2023-06-23T18:23:38.200 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb 2023-06-23T18:23:38.201 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} --owner scheduled_rishabh@teuthology --name rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi -- /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886/orig.config.yaml 2023-06-23T18:23:38.252 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-23T18:23:38.343 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886 branch: wip-rishabh-improvements-authmon description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} email: null first_in_suite: false job_id: '7313886' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-improvements-authmon ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-improvements-authmon sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-improvements-authmon suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi079.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_quota teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 076bbebc42a14f7d568aaa78eabb0038327bcb23 timestamp: 2023-06-23_17:37:30 tube: smithi user: rishabh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-06-23T18:23:38.343 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa; will attempt to use it 2023-06-23T18:23:38.344 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks 2023-06-23T18:23:38.344 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-06-23T18:23:38.346 INFO:teuthology.task.internal:Checking packages... 2023-06-23T18:23:38.374 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '8627af3e0adcb765a3c249fcc209cba9f4873e1b' 2023-06-23T18:23:38.375 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-06-23T18:23:38.375 INFO:teuthology.packaging:ref: None 2023-06-23T18:23:38.375 INFO:teuthology.packaging:tag: None 2023-06-23T18:23:38.376 INFO:teuthology.packaging:branch: wip-rishabh-improvements-authmon 2023-06-23T18:23:38.376 INFO:teuthology.packaging:sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:23:38.376 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-improvements-authmon 2023-06-23T18:23:38.523 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4479.g8627af3e 2023-06-23T18:23:38.523 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-06-23T18:23:38.530 INFO:teuthology.task.internal:no buildpackages task found 2023-06-23T18:23:38.530 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-06-23T18:23:38.537 INFO:teuthology.task.internal:Saving configuration 2023-06-23T18:23:38.559 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-06-23T18:23:38.565 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T18:23:38.596 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 18:18:21.501393', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-23T18:23:38.624 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 18:18:21.503336', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-23T18:23:38.624 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-06-23T18:23:38.631 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-06-23T18:23:38.631 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-06-23T18:23:38.631 INFO:teuthology.run_tasks:Running task console_log... 2023-06-23T18:23:38.707 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6562204040>, signals=[15]) 2023-06-23T18:23:38.708 INFO:teuthology.run_tasks:Running task internal.connect... 2023-06-23T18:23:38.715 INFO:teuthology.task.internal:Opening connections... 2023-06-23T18:23:38.715 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:23:38.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T18:23:38.787 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:23:38.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T18:23:38.853 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-06-23T18:23:38.860 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2023-06-23T18:23:38.881 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2023-06-23T18:23:38.882 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2023-06-23T18:23:38.940 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="Red Hat Enterprise Linux" 2023-06-23T18:23:38.941 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="8.6 (Ootpa)" 2023-06-23T18:23:38.941 INFO:teuthology.orchestra.run.smithi079.stdout:ID="rhel" 2023-06-23T18:23:38.941 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="fedora" 2023-06-23T18:23:38.941 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="8.6" 2023-06-23T18:23:38.941 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el8" 2023-06-23T18:23:38.941 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-23T18:23:38.942 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2023-06-23T18:23:38.942 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-23T18:23:38.942 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://www.redhat.com/" 2023-06-23T18:23:38.942 INFO:teuthology.orchestra.run.smithi079.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-23T18:23:38.942 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-23T18:23:38.942 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:23:38.943 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-23T18:23:38.943 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-23T18:23:38.943 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-23T18:23:38.943 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-23T18:23:38.944 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2023-06-23T18:23:38.974 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2023-06-23T18:23:38.995 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2023-06-23T18:23:38.995 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2023-06-23T18:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Red Hat Enterprise Linux" 2023-06-23T18:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="8.6 (Ootpa)" 2023-06-23T18:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout:ID="rhel" 2023-06-23T18:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="fedora" 2023-06-23T18:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="8.6" 2023-06-23T18:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el8" 2023-06-23T18:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-23T18:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2023-06-23T18:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-23T18:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.redhat.com/" 2023-06-23T18:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-23T18:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-23T18:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-23T18:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-23T18:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-23T18:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-23T18:23:39.055 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2023-06-23T18:23:39.082 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-06-23T18:23:39.092 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-06-23T18:23:39.098 INFO:teuthology.task.internal:Checking for old test directory... 2023-06-23T18:23:39.098 DEBUG:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2023-06-23T18:23:39.101 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2023-06-23T18:23:39.119 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-06-23T18:23:39.137 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-06-23T18:23:39.138 DEBUG:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2023-06-23T18:23:39.161 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2023-06-23T18:23:39.188 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-06-23T18:23:39.246 INFO:teuthology.run_tasks:Running task kernel... 2023-06-23T18:23:39.263 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-06-23T18:23:39.264 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-06-23T18:23:39.265 DEBUG:teuthology.orchestra.run.smithi079:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-23T18:23:39.266 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-23T18:23:39.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:23:39.284 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2023-06-23T18:23:39.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:23:39.286 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2023-06-23T18:23:39.343 INFO:teuthology.orchestra.run.smithi079.stdout:4.18.0-372.9.1.el8.x86_64 2023-06-23T18:23:39.343 INFO:teuthology.task.kernel:Running kernel on smithi079: 4.18.0-372.9.1.el8.x86_64 2023-06-23T18:23:39.343 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum install -y kernel 2023-06-23T18:23:39.345 INFO:teuthology.orchestra.run.smithi155.stdout:4.18.0-372.9.1.el8.x86_64 2023-06-23T18:23:39.345 INFO:teuthology.task.kernel:Running kernel on smithi155: 4.18.0-372.9.1.el8.x86_64 2023-06-23T18:23:39.345 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum install -y kernel 2023-06-23T18:23:40.544 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2023-06-23T18:23:40.545 INFO:teuthology.orchestra.run.smithi079.stdout:Unable to read consumer identity 2023-06-23T18:23:40.575 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2023-06-23T18:23:40.576 INFO:teuthology.orchestra.run.smithi155.stdout:Unable to read consumer identity 2023-06-23T18:23:44.204 INFO:teuthology.orchestra.run.smithi079.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-23T18:23:44.455 INFO:teuthology.orchestra.run.smithi155.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-06-23T18:23:47.136 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 5.4 MB/s | 16 MB 00:02 2023-06-23T18:23:47.397 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 5.3 MB/s | 16 MB 00:02 2023-06-23T18:23:50.617 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 301 kB/s | 24 kB 00:00 2023-06-23T18:23:50.797 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 528 kB/s | 24 kB 00:00 2023-06-23T18:23:51.297 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-06-23T18:23:51.347 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-06-23T18:23:51.348 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2023-06-23T18:23:51.348 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-06-23T18:23:51.464 DEBUG:teuthology.orchestra.run.smithi079:> echo no | sudo yum reinstall kernel || true 2023-06-23T18:23:51.499 INFO:teuthology.orchestra.run.smithi155.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-06-23T18:23:51.561 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2023-06-23T18:23:51.563 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2023-06-23T18:23:51.563 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2023-06-23T18:23:51.689 DEBUG:teuthology.orchestra.run.smithi155:> echo no | sudo yum reinstall kernel || true 2023-06-23T18:23:51.794 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2023-06-23T18:23:51.795 INFO:teuthology.orchestra.run.smithi079.stdout:Unable to read consumer identity 2023-06-23T18:23:52.055 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:02 ago on Fri 23 Jun 2023 06:23:50 PM UTC. 2023-06-23T18:23:52.060 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2023-06-23T18:23:52.061 INFO:teuthology.orchestra.run.smithi155.stdout:Unable to read consumer identity 2023-06-23T18:23:52.096 INFO:teuthology.orchestra.run.smithi079.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-23T18:23:52.135 INFO:teuthology.orchestra.run.smithi079.stderr:Error: No packages marked for reinstall. 2023-06-23T18:23:52.180 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum reinstall -y kernel || true 2023-06-23T18:23:52.321 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:02 ago on Fri 23 Jun 2023 06:23:50 PM UTC. 2023-06-23T18:23:52.361 INFO:teuthology.orchestra.run.smithi155.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-23T18:23:52.393 INFO:teuthology.orchestra.run.smithi155.stderr:Error: No packages marked for reinstall. 2023-06-23T18:23:52.439 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum reinstall -y kernel || true 2023-06-23T18:23:52.508 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2023-06-23T18:23:52.509 INFO:teuthology.orchestra.run.smithi079.stdout:Unable to read consumer identity 2023-06-23T18:23:52.758 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:02 ago on Fri 23 Jun 2023 06:23:50 PM UTC. 2023-06-23T18:23:52.768 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2023-06-23T18:23:52.768 INFO:teuthology.orchestra.run.smithi155.stdout:Unable to read consumer identity 2023-06-23T18:23:52.798 INFO:teuthology.orchestra.run.smithi079.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-23T18:23:52.832 INFO:teuthology.orchestra.run.smithi079.stderr:Error: No packages marked for reinstall. 2023-06-23T18:23:52.876 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q kernel | sort -rV | head -n 1 2023-06-23T18:23:52.956 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-06-23T18:23:52.957 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-06-23T18:23:52.957 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-06-23T18:23:52.957 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-06-23T18:23:52.957 DEBUG:teuthology.orchestra.run.smithi079:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-23T18:23:52.983 INFO:teuthology.orchestra.run.smithi079.stdout:ttyS1 2023-06-23T18:23:53.003 DEBUG:teuthology.parallel:result is None 2023-06-23T18:23:53.018 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:03 ago on Fri 23 Jun 2023 06:23:50 PM UTC. 2023-06-23T18:23:53.060 INFO:teuthology.orchestra.run.smithi155.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-23T18:23:53.086 INFO:teuthology.orchestra.run.smithi155.stderr:Error: No packages marked for reinstall. 2023-06-23T18:23:53.132 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q kernel | sort -rV | head -n 1 2023-06-23T18:23:53.265 INFO:teuthology.orchestra.run.smithi155.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-06-23T18:23:53.265 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-06-23T18:23:53.265 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-06-23T18:23:53.266 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-06-23T18:23:53.266 DEBUG:teuthology.orchestra.run.smithi155:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-23T18:23:53.292 INFO:teuthology.orchestra.run.smithi155.stdout:ttyS1 2023-06-23T18:23:53.312 DEBUG:teuthology.parallel:result is None 2023-06-23T18:23:53.313 INFO:teuthology.run_tasks:Running task internal.base... 2023-06-23T18:23:53.320 INFO:teuthology.task.internal:Creating test directory... 2023-06-23T18:23:53.321 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-23T18:23:53.323 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-23T18:23:53.369 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-06-23T18:23:53.375 INFO:teuthology.run_tasks:Running task internal.archive... 2023-06-23T18:23:53.385 INFO:teuthology.task.internal:Creating archive directory... 2023-06-23T18:23:53.386 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-23T18:23:53.388 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-23T18:23:53.430 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-06-23T18:23:53.438 INFO:teuthology.task.internal:Enabling coredump saving... 2023-06-23T18:23:53.438 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-23T18:23:53.461 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-23T18:23:53.485 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T18:23:53.496 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T18:23:53.507 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T18:23:53.518 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T18:23:53.519 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-06-23T18:23:53.526 INFO:teuthology.task.internal:Configuring sudo... 2023-06-23T18:23:53.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-23T18:23:53.540 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-23T18:23:53.586 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-06-23T18:23:53.595 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-06-23T18:23:53.596 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-23T18:23:53.606 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-23T18:23:53.643 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T18:23:53.707 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T18:23:53.760 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T18:23:53.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T18:23:53.883 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:23:53.884 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-23T18:23:53.950 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T18:23:53.999 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T18:23:54.043 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T18:23:54.129 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T18:23:54.169 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:23:54.169 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-23T18:23:54.236 DEBUG:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2023-06-23T18:23:54.238 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2023-06-23T18:23:54.305 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T18:23:54.336 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T18:23:54.692 INFO:teuthology.run_tasks:Running task internal.timer... 2023-06-23T18:23:54.704 INFO:teuthology.task.internal:Starting timer... 2023-06-23T18:23:54.704 INFO:teuthology.run_tasks:Running task pcp... 2023-06-23T18:23:54.718 INFO:teuthology.run_tasks:Running task selinux... 2023-06-23T18:23:54.731 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-06-23T18:23:54.732 DEBUG:teuthology.orchestra.run.smithi079:> sudo service auditd rotate 2023-06-23T18:23:54.790 INFO:teuthology.orchestra.run.smithi079.stdout:Rotating logs: 2023-06-23T18:23:54.792 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2023-06-23T18:23:54.858 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2023-06-23T18:23:54.859 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-06-23T18:23:54.860 DEBUG:teuthology.orchestra.run.smithi079:> /usr/sbin/getenforce 2023-06-23T18:23:54.884 INFO:teuthology.orchestra.run.smithi079.stdout:Permissive 2023-06-23T18:23:54.885 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2023-06-23T18:23:54.904 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2023-06-23T18:23:54.905 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi079.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive'} 2023-06-23T18:23:54.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T18:23:54.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:23:54.953 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T18:23:54.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:23:54.980 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-06-23T18:23:54.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo /usr/sbin/setenforce permissive 2023-06-23T18:23:55.021 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2023-06-23T18:23:55.048 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-06-23T18:23:55.062 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-06-23T18:23:55.063 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-06-23T18:23:55.084 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-06-23T18:23:55.086 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi079.front.sepia.ceph.com,smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-06-23T18:32:24.193 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi079.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2023-06-23T18:32:24.194 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2023-06-23T18:32:24.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T18:32:24.265 DEBUG:teuthology.orchestra.run.smithi079:> true 2023-06-23T18:32:24.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-06-23T18:32:24.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T18:32:24.405 DEBUG:teuthology.orchestra.run.smithi155:> true 2023-06-23T18:32:24.473 INFO:teuthology.run_tasks:Running task clock... 2023-06-23T18:32:24.491 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-06-23T18:32:24.492 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-23T18:32:24.493 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T18:32:24.495 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-23T18:32:24.496 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T18:32:24.527 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-23T18:32:24.541 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-23T18:32:24.544 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-23T18:32:24.558 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-23T18:32:24.608 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: ntpd: command not found 2023-06-23T18:32:24.620 INFO:teuthology.orchestra.run.smithi079.stdout:506 Cannot talk to daemon 2023-06-23T18:32:24.630 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2023-06-23T18:32:24.634 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-23T18:32:24.641 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2023-06-23T18:32:24.648 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-23T18:32:24.656 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-23T18:32:24.669 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-23T18:32:24.720 INFO:teuthology.orchestra.run.smithi079.stderr:bash: ntpq: command not found 2023-06-23T18:32:24.724 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T18:32:24.725 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2023-06-23T18:32:24.725 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:32:24.725 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:32:24.725 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:32:24.725 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:32:24.739 INFO:teuthology.orchestra.run.smithi155.stderr:bash: ntpq: command not found 2023-06-23T18:32:24.743 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T18:32:24.743 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2023-06-23T18:32:24.743 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:32:24.743 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:32:24.743 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:32:24.743 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:32:24.744 INFO:teuthology.run_tasks:Running task install... 2023-06-23T18:32:24.756 DEBUG:teuthology.task.install:project ceph 2023-06-23T18:32:24.757 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b'}} 2023-06-23T18:32:24.757 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b'} 2023-06-23T18:32:24.758 INFO:teuthology.task.install:Using flavor: default 2023-06-23T18:32:24.763 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-06-23T18:32:24.764 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-06-23T18:32:24.764 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b', 'tag': None, 'wait_for_package': False} 2023-06-23T18:32:24.765 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:32:24.767 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b', 'tag': None, 'wait_for_package': False} 2023-06-23T18:32:24.768 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:32:25.057 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/ 2023-06-23T18:32:25.058 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4479.g8627af3e 2023-06-23T18:32:25.065 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/ 2023-06-23T18:32:25.066 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4479.g8627af3e 2023-06-23T18:32:25.302 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-23T18:32:25.302 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:32:25.302 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-23T18:32:25.315 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-23T18:32:25.316 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:32:25.316 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-23T18:32:25.332 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-06-23T18:32:25.333 DEBUG:teuthology.orchestra.run.smithi079:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8627af3e0adcb765a3c249fcc209cba9f4873e1b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-23T18:32:25.346 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-06-23T18:32:25.346 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8627af3e0adcb765a3c249fcc209cba9f4873e1b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-23T18:32:25.401 DEBUG:teuthology.orchestra.run.smithi079:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-23T18:32:25.415 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-23T18:32:25.479 DEBUG:teuthology.orchestra.run.smithi079:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-23T18:32:25.493 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-23T18:32:25.546 INFO:teuthology.orchestra.run.smithi079.stdout:check_obsoletes = 1 2023-06-23T18:32:25.547 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum clean all 2023-06-23T18:32:25.559 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2023-06-23T18:32:25.560 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2023-06-23T18:32:25.918 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2023-06-23T18:32:25.935 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2023-06-23T18:32:26.180 INFO:teuthology.orchestra.run.smithi079.stdout:44 files removed 2023-06-23T18:32:26.188 INFO:teuthology.orchestra.run.smithi155.stdout:44 files removed 2023-06-23T18:32:26.225 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-06-23T18:32:26.236 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-06-23T18:32:26.554 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2023-06-23T18:32:26.570 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2023-06-23T18:32:27.179 INFO:teuthology.orchestra.run.smithi079.stdout:ceph packages for x86_64 314 kB/s | 81 kB 00:00 2023-06-23T18:32:27.549 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 131 kB/s | 81 kB 00:00 2023-06-23T18:32:27.587 INFO:teuthology.orchestra.run.smithi079.stdout:ceph noarch packages 25 kB/s | 9.4 kB 00:00 2023-06-23T18:32:27.949 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 26 kB/s | 9.4 kB 00:00 2023-06-23T18:32:27.971 INFO:teuthology.orchestra.run.smithi079.stdout:ceph source packages 4.1 kB/s | 1.5 kB 00:00 2023-06-23T18:32:28.308 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 4.4 kB/s | 1.5 kB 00:00 2023-06-23T18:32:30.705 INFO:teuthology.orchestra.run.smithi079.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-06-23T18:32:31.000 INFO:teuthology.orchestra.run.smithi155.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-06-23T18:32:31.851 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 19 MB/s | 16 MB 00:00 2023-06-23T18:32:35.280 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 297 kB/s | 24 kB 00:00 2023-06-23T18:32:36.186 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-06-23T18:32:44.228 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-06-23T18:32:50.088 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 826 kB/s | 16 MB 00:19 2023-06-23T18:32:53.628 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 529 kB/s | 24 kB 00:00 2023-06-23T18:32:54.579 INFO:teuthology.orchestra.run.smithi079.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 60 MB/s | 53 MB 00:00 2023-06-23T18:32:55.124 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T18:32:55.125 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T18:32:55.139 INFO:teuthology.orchestra.run.smithi155.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-06-23T18:32:55.140 INFO:teuthology.orchestra.run.smithi155.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-06-23T18:32:55.141 INFO:teuthology.orchestra.run.smithi155.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-06-23T18:32:55.142 INFO:teuthology.orchestra.run.smithi155.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-23T18:32:55.145 INFO:teuthology.orchestra.run.smithi155.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-23T18:32:55.148 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-23T18:32:55.149 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-23T18:32:55.278 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2023-06-23T18:32:55.291 INFO:teuthology.orchestra.run.smithi155.stdout:=========================================================================================================================== 2023-06-23T18:32:55.291 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2023-06-23T18:32:55.291 INFO:teuthology.orchestra.run.smithi155.stdout:=========================================================================================================================== 2023-06-23T18:32:55.292 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2023-06-23T18:32:55.292 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-06-23T18:32:55.292 INFO:teuthology.orchestra.run.smithi155.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-23T18:32:55.293 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 6.4 k 2023-06-23T18:32:55.293 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 5.4 M 2023-06-23T18:32:55.293 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 848 k 2023-06-23T18:32:55.294 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 165 k 2023-06-23T18:32:55.294 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 1.5 M 2023-06-23T18:32:55.294 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 128 k 2023-06-23T18:32:55.294 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 1.7 M 2023-06-23T18:32:55.295 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 7.4 M 2023-06-23T18:32:55.295 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 47 k 2023-06-23T18:32:55.295 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 13 M 2023-06-23T18:32:55.296 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 48 M 2023-06-23T18:32:55.296 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 258 k 2023-06-23T18:32:55.296 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 161 k 2023-06-23T18:32:55.296 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 258 k 2023-06-23T18:32:55.296 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 18 k 2023-06-23T18:32:55.297 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-06-23T18:32:55.297 INFO:teuthology.orchestra.run.smithi155.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-23T18:32:55.297 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-23T18:32:55.297 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 31 k 2023-06-23T18:32:55.297 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 771 k 2023-06-23T18:32:55.297 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 139 k 2023-06-23T18:32:55.297 INFO:teuthology.orchestra.run.smithi155.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-23T18:32:55.297 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-06-23T18:32:55.298 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-23T18:32:55.298 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-06-23T18:32:55.298 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-06-23T18:32:55.298 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-06-23T18:32:55.298 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 197 k 2023-06-23T18:32:55.298 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 370 k 2023-06-23T18:32:55.299 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 365 k 2023-06-23T18:32:55.299 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 107 k 2023-06-23T18:32:55.299 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 94 k 2023-06-23T18:32:55.299 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.4 M 2023-06-23T18:32:55.299 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 179 k 2023-06-23T18:32:55.300 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-06-23T18:32:55.300 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2023-06-23T18:32:55.300 INFO:teuthology.orchestra.run.smithi155.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-06-23T18:32:55.300 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.7 M 2023-06-23T18:32:55.300 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.6 M 2023-06-23T18:32:55.300 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2023-06-23T18:32:55.301 INFO:teuthology.orchestra.run.smithi155.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-23T18:32:55.301 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 22 M 2023-06-23T18:32:55.301 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 23 k 2023-06-23T18:32:55.301 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 2.3 M 2023-06-23T18:32:55.301 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 240 k 2023-06-23T18:32:55.301 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 4.6 M 2023-06-23T18:32:55.301 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 18 M 2023-06-23T18:32:55.302 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 15 k 2023-06-23T18:32:55.302 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 24 k 2023-06-23T18:32:55.302 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-23T18:32:55.302 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-23T18:32:55.302 INFO:teuthology.orchestra.run.smithi155.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-06-23T18:32:55.302 INFO:teuthology.orchestra.run.smithi155.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-23T18:32:55.302 INFO:teuthology.orchestra.run.smithi155.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-06-23T18:32:55.303 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 137 k 2023-06-23T18:32:55.303 INFO:teuthology.orchestra.run.smithi155.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-06-23T18:32:55.303 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-06-23T18:32:55.303 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-23T18:32:55.303 INFO:teuthology.orchestra.run.smithi155.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-06-23T18:32:55.303 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-06-23T18:32:55.303 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-23T18:32:55.304 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 474 k 2023-06-23T18:32:55.304 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-06-23T18:32:55.304 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 6.7 M 2023-06-23T18:32:55.304 INFO:teuthology.orchestra.run.smithi155.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-23T18:32:55.304 INFO:teuthology.orchestra.run.smithi155.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-06-23T18:32:55.304 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-23T18:32:55.305 INFO:teuthology.orchestra.run.smithi155.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-23T18:32:55.305 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-23T18:32:55.305 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-23T18:32:55.305 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-23T18:32:55.305 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-06-23T18:32:55.305 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-23T18:32:55.305 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-23T18:32:55.306 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-23T18:32:55.306 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-23T18:32:55.306 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-23T18:32:55.306 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-23T18:32:55.306 INFO:teuthology.orchestra.run.smithi155.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-23T18:32:55.306 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-23T18:32:55.306 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 45 k 2023-06-23T18:32:55.307 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 119 k 2023-06-23T18:32:55.307 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-23T18:32:55.307 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-06-23T18:32:55.307 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-23T18:32:55.307 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-23T18:32:55.307 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-23T18:32:55.307 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-23T18:32:55.308 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-23T18:32:55.308 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-23T18:32:55.308 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-23T18:32:55.308 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-23T18:32:55.308 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-23T18:32:55.308 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T18:32:55.308 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-23T18:32:55.309 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-23T18:32:55.309 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-23T18:32:55.309 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-23T18:32:55.309 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-23T18:32:55.309 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-23T18:32:55.309 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-23T18:32:55.309 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-23T18:32:55.309 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-23T18:32:55.310 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-23T18:32:55.310 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-23T18:32:55.310 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-23T18:32:55.310 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-06-23T18:32:55.310 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-23T18:32:55.310 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-23T18:32:55.310 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-06-23T18:32:55.310 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-23T18:32:55.311 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-23T18:32:55.311 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T18:32:55.311 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-23T18:32:55.311 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-23T18:32:55.311 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-06-23T18:32:55.311 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-23T18:32:55.311 INFO:teuthology.orchestra.run.smithi155.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-23T18:32:55.312 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-23T18:32:55.312 INFO:teuthology.orchestra.run.smithi155.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-23T18:32:55.312 INFO:teuthology.orchestra.run.smithi155.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-23T18:32:55.312 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-23T18:32:55.312 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-23T18:32:55.312 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-23T18:32:55.312 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-23T18:32:55.313 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-23T18:32:55.313 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-06-23T18:32:55.313 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-23T18:32:55.313 INFO:teuthology.orchestra.run.smithi155.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-06-23T18:32:55.313 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-23T18:32:55.313 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-23T18:32:55.313 INFO:teuthology.orchestra.run.smithi155.stdout:Enabling module streams: 2023-06-23T18:32:55.313 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql 10 2023-06-23T18:32:55.314 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:32:55.314 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2023-06-23T18:32:55.314 INFO:teuthology.orchestra.run.smithi155.stdout:=========================================================================================================================== 2023-06-23T18:32:55.314 INFO:teuthology.orchestra.run.smithi155.stdout:Install 128 Packages 2023-06-23T18:32:55.314 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 3 Packages 2023-06-23T18:32:55.314 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:32:55.315 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 205 M 2023-06-23T18:32:55.315 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2023-06-23T18:32:55.628 INFO:teuthology.orchestra.run.smithi155.stdout:(1/131): ceph-18.0.0-4479.g8627af3e.el8.x86_64. 55 kB/s | 6.4 kB 00:00 2023-06-23T18:32:55.820 INFO:teuthology.orchestra.run.smithi155.stdout:(2/131): ceph-fuse-18.0.0-4479.g8627af3e.el8.x8 4.3 MB/s | 848 kB 00:00 2023-06-23T18:32:55.879 INFO:teuthology.orchestra.run.smithi155.stdout:(3/131): ceph-immutable-object-cache-18.0.0-447 2.8 MB/s | 165 kB 00:00 2023-06-23T18:32:56.213 INFO:teuthology.orchestra.run.smithi155.stdout:(4/131): ceph-mds-18.0.0-4479.g8627af3e.el8.x86 6.9 MB/s | 2.3 MB 00:00 2023-06-23T18:32:56.288 INFO:teuthology.orchestra.run.smithi155.stdout:(5/131): ceph-base-18.0.0-4479.g8627af3e.el8.x8 7.0 MB/s | 5.4 MB 00:00 2023-06-23T18:32:56.446 INFO:teuthology.orchestra.run.smithi155.stdout:(6/131): ceph-mgr-18.0.0-4479.g8627af3e.el8.x86 6.6 MB/s | 1.5 MB 00:00 2023-06-23T18:32:56.847 INFO:teuthology.orchestra.run.smithi155.stdout:(7/131): ceph-mon-18.0.0-4479.g8627af3e.el8.x86 8.3 MB/s | 4.6 MB 00:00 2023-06-23T18:32:57.830 INFO:teuthology.orchestra.run.smithi155.stdout:(8/131): ceph-common-18.0.0-4479.g8627af3e.el8. 9.5 MB/s | 22 MB 00:02 2023-06-23T18:32:57.905 INFO:teuthology.orchestra.run.smithi155.stdout:(9/131): ceph-selinux-18.0.0-4479.g8627af3e.el8 326 kB/s | 24 kB 00:00 2023-06-23T18:32:58.423 INFO:teuthology.orchestra.run.smithi155.stdout:(10/131): ceph-radosgw-18.0.0-4479.g8627af3e.el 8.6 MB/s | 13 MB 00:01 2023-06-23T18:32:58.639 INFO:teuthology.orchestra.run.smithi155.stdout:(11/131): ceph-osd-18.0.0-4479.g8627af3e.el8.x8 8.1 MB/s | 18 MB 00:02 2023-06-23T18:32:58.665 INFO:teuthology.orchestra.run.smithi155.stdout:(12/131): cephfs-mirror-18.0.0-4479.g8627af3e.e 1.0 MB/s | 258 kB 00:00 2023-06-23T18:32:58.682 INFO:teuthology.orchestra.run.smithi155.stdout:(13/131): libcephfs-devel-18.0.0-4479.g8627af3e 743 kB/s | 31 kB 00:00 2023-06-23T18:32:58.724 INFO:teuthology.orchestra.run.smithi155.stdout:(14/131): libcephsqlite-18.0.0-4479.g8627af3e.e 3.2 MB/s | 137 kB 00:00 2023-06-23T18:32:58.780 INFO:teuthology.orchestra.run.smithi155.stdout:(15/131): libcephfs2-18.0.0-4479.g8627af3e.el8. 6.5 MB/s | 771 kB 00:00 2023-06-23T18:32:58.808 INFO:teuthology.orchestra.run.smithi155.stdout:(16/131): librados-devel-18.0.0-4479.g8627af3e. 1.6 MB/s | 139 kB 00:00 2023-06-23T18:32:58.841 INFO:teuthology.orchestra.run.smithi155.stdout:(17/131): libradosstriper1-18.0.0-4479.g8627af3 7.7 MB/s | 474 kB 00:00 2023-06-23T18:32:58.908 INFO:teuthology.orchestra.run.smithi155.stdout:(18/131): python3-ceph-argparse-18.0.0-4479.g86 671 kB/s | 45 kB 00:00 2023-06-23T18:32:58.983 INFO:teuthology.orchestra.run.smithi155.stdout:(19/131): python3-ceph-common-18.0.0-4479.g8627 1.6 MB/s | 119 kB 00:00 2023-06-23T18:32:59.067 INFO:teuthology.orchestra.run.smithi155.stdout:(20/131): python3-cephfs-18.0.0-4479.g8627af3e. 2.3 MB/s | 197 kB 00:00 2023-06-23T18:32:59.176 INFO:teuthology.orchestra.run.smithi155.stdout:(21/131): python3-rados-18.0.0-4479.g8627af3e.e 3.3 MB/s | 370 kB 00:00 2023-06-23T18:32:59.267 INFO:teuthology.orchestra.run.smithi155.stdout:(22/131): python3-rbd-18.0.0-4479.g8627af3e.el8 3.9 MB/s | 365 kB 00:00 2023-06-23T18:32:59.351 INFO:teuthology.orchestra.run.smithi155.stdout:(23/131): python3-rgw-18.0.0-4479.g8627af3e.el8 1.2 MB/s | 107 kB 00:00 2023-06-23T18:32:59.435 INFO:teuthology.orchestra.run.smithi155.stdout:(24/131): rbd-fuse-18.0.0-4479.g8627af3e.el8.x8 1.1 MB/s | 94 kB 00:00 2023-06-23T18:32:59.593 INFO:teuthology.orchestra.run.smithi155.stdout:(25/131): librgw2-18.0.0-4479.g8627af3e.el8.x86 8.6 MB/s | 6.7 MB 00:00 2023-06-23T18:32:59.669 INFO:teuthology.orchestra.run.smithi155.stdout:(26/131): rbd-nbd-18.0.0-4479.g8627af3e.el8.x86 2.3 MB/s | 179 kB 00:00 2023-06-23T18:32:59.744 INFO:teuthology.orchestra.run.smithi155.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4479.g 308 kB/s | 23 kB 00:00 2023-06-23T18:32:59.811 INFO:teuthology.orchestra.run.smithi155.stdout:(28/131): rbd-mirror-18.0.0-4479.g8627af3e.el8. 9.2 MB/s | 3.4 MB 00:00 2023-06-23T18:32:59.844 INFO:teuthology.orchestra.run.smithi155.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4479.g8627af3 1.2 MB/s | 128 kB 00:00 2023-06-23T18:33:00.045 INFO:teuthology.orchestra.run.smithi155.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4479.g8627a 7.3 MB/s | 1.7 MB 00:00 2023-06-23T18:33:00.129 INFO:teuthology.orchestra.run.smithi155.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4479.g86 2.8 MB/s | 240 kB 00:00 2023-06-23T18:33:00.204 INFO:teuthology.orchestra.run.smithi155.stdout:(32/131): ceph-mgr-rook-18.0.0-4479.g8627af3e.e 632 kB/s | 47 kB 00:00 2023-06-23T18:33:00.279 INFO:teuthology.orchestra.run.smithi155.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4479.g8 194 kB/s | 15 kB 00:00 2023-06-23T18:33:00.363 INFO:teuthology.orchestra.run.smithi155.stdout:(34/131): ceph-volume-18.0.0-4479.g8627af3e.el8 3.0 MB/s | 258 kB 00:00 2023-06-23T18:33:00.446 INFO:teuthology.orchestra.run.smithi155.stdout:(35/131): cephadm-18.0.0-4479.g8627af3e.el8.noa 1.9 MB/s | 161 kB 00:00 2023-06-23T18:33:00.513 INFO:teuthology.orchestra.run.smithi155.stdout:(36/131): cephfs-top-18.0.0-4479.g8627af3e.el8. 269 kB/s | 18 kB 00:00 2023-06-23T18:33:00.680 INFO:teuthology.orchestra.run.smithi155.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 8.8 MB/s | 7.4 MB 00:00 2023-06-23T18:33:00.722 INFO:teuthology.orchestra.run.smithi155.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2023-06-23T18:33:00.764 INFO:teuthology.orchestra.run.smithi155.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 2.0 MB/s | 75 kB 00:00 2023-06-23T18:33:00.823 INFO:teuthology.orchestra.run.smithi155.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 8.1 MB/s | 476 kB 00:00 2023-06-23T18:33:00.848 INFO:teuthology.orchestra.run.smithi155.stdout:(41/131): liboath-2.6.2-3.el8.x86_64.rpm 351 kB/s | 59 kB 00:00 2023-06-23T18:33:00.873 INFO:teuthology.orchestra.run.smithi155.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 866 kB/s | 44 kB 00:00 2023-06-23T18:33:01.307 INFO:teuthology.orchestra.run.smithi155.stdout:(43/131): ceph-test-18.0.0-4479.g8627af3e.el8.x 14 MB/s | 48 MB 00:03 2023-06-23T18:33:01.370 INFO:teuthology.orchestra.run.smithi155.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 66 kB/s | 33 kB 00:00 2023-06-23T18:33:01.408 INFO:teuthology.orchestra.run.smithi155.stdout:(45/131): python3-beautifulsoup4-4.6.3-2.el8.1. 331 kB/s | 185 kB 00:00 2023-06-23T18:33:01.433 INFO:teuthology.orchestra.run.smithi155.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.9 MB/s | 173 kB 00:00 2023-06-23T18:33:01.467 INFO:teuthology.orchestra.run.smithi155.stdout:(47/131): python3-certifi-2018.10.15-7.el8.noar 103 kB/s | 16 kB 00:00 2023-06-23T18:33:01.492 INFO:teuthology.orchestra.run.smithi155.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-06-23T18:33:01.517 INFO:teuthology.orchestra.run.smithi155.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 222 kB/s | 11 kB 00:00 2023-06-23T18:33:01.534 INFO:teuthology.orchestra.run.smithi155.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 421 kB/s | 18 kB 00:00 2023-06-23T18:33:01.559 INFO:teuthology.orchestra.run.smithi155.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.5 MB/s | 384 kB 00:00 2023-06-23T18:33:01.584 INFO:teuthology.orchestra.run.smithi155.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 971 kB/s | 49 kB 00:00 2023-06-23T18:33:01.609 INFO:teuthology.orchestra.run.smithi155.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-06-23T18:33:01.635 INFO:teuthology.orchestra.run.smithi155.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-06-23T18:33:01.685 INFO:teuthology.orchestra.run.smithi155.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 312 kB/s | 16 kB 00:00 2023-06-23T18:33:01.769 INFO:teuthology.orchestra.run.smithi155.stdout:(56/131): python3-kubernetes-11.0.0-6.el8.noarc 4.6 MB/s | 1.2 MB 00:00 2023-06-23T18:33:01.794 INFO:teuthology.orchestra.run.smithi155.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 309 kB/s | 34 kB 00:00 2023-06-23T18:33:01.819 INFO:teuthology.orchestra.run.smithi155.stdout:(58/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2023-06-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi155.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2023-06-23T18:33:01.870 INFO:teuthology.orchestra.run.smithi155.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 374 kB/s | 19 kB 00:00 2023-06-23T18:33:01.895 INFO:teuthology.orchestra.run.smithi155.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 479 kB/s | 24 kB 00:00 2023-06-23T18:33:01.920 INFO:teuthology.orchestra.run.smithi155.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 582 kB/s | 29 kB 00:00 2023-06-23T18:33:01.954 INFO:teuthology.orchestra.run.smithi155.stdout:(63/131): python3-routes-2.4.1-12.el8.noarch.rp 1.0 MB/s | 196 kB 00:00 2023-06-23T18:33:01.979 INFO:teuthology.orchestra.run.smithi155.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 329 kB/s | 27 kB 00:00 2023-06-23T18:33:02.004 INFO:teuthology.orchestra.run.smithi155.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-06-23T18:33:02.030 INFO:teuthology.orchestra.run.smithi155.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 2.3 MB/s | 255 kB 00:00 2023-06-23T18:33:02.055 INFO:teuthology.orchestra.run.smithi155.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 814 kB/s | 61 kB 00:00 2023-06-23T18:33:02.100 INFO:teuthology.orchestra.run.smithi155.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 328 kB/s | 23 kB 00:00 2023-06-23T18:33:02.131 INFO:teuthology.orchestra.run.smithi155.stdout:(69/131): python3-webtest-2.0.33-1.el8.noarch.r 678 kB/s | 85 kB 00:00 2023-06-23T18:33:02.156 INFO:teuthology.orchestra.run.smithi155.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2023-06-23T18:33:02.206 INFO:teuthology.orchestra.run.smithi155.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 624 kB/s | 47 kB 00:00 2023-06-23T18:33:02.248 INFO:teuthology.orchestra.run.smithi155.stdout:(72/131): thrift-0.13.0-2.el8.x86_64.rpm 9.0 MB/s | 1.7 MB 00:00 2023-06-23T18:33:02.273 INFO:teuthology.orchestra.run.smithi155.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 637 kB/s | 43 kB 00:00 2023-06-23T18:33:02.307 INFO:teuthology.orchestra.run.smithi155.stdout:(74/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.3 MB/s | 200 kB 00:00 2023-06-23T18:33:02.332 INFO:teuthology.orchestra.run.smithi155.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 908 kB/s | 48 kB 00:00 2023-06-23T18:33:02.357 INFO:teuthology.orchestra.run.smithi155.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-06-23T18:33:02.383 INFO:teuthology.orchestra.run.smithi155.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 851 kB/s | 43 kB 00:00 2023-06-23T18:33:02.408 INFO:teuthology.orchestra.run.smithi155.stdout:(78/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2023-06-23T18:33:02.433 INFO:teuthology.orchestra.run.smithi155.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-06-23T18:33:02.492 INFO:teuthology.orchestra.run.smithi155.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2023-06-23T18:33:02.517 INFO:teuthology.orchestra.run.smithi155.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 224 kB/s | 19 kB 00:00 2023-06-23T18:33:02.543 INFO:teuthology.orchestra.run.smithi155.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-06-23T18:33:02.584 INFO:teuthology.orchestra.run.smithi155.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.1 MB/s | 560 kB 00:00 2023-06-23T18:33:02.609 INFO:teuthology.orchestra.run.smithi155.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 269 kB/s | 18 kB 00:00 2023-06-23T18:33:02.643 INFO:teuthology.orchestra.run.smithi155.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.3 MB/s | 559 kB 00:00 2023-06-23T18:33:02.668 INFO:teuthology.orchestra.run.smithi155.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-06-23T18:33:02.694 INFO:teuthology.orchestra.run.smithi155.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-06-23T18:33:02.719 INFO:teuthology.orchestra.run.smithi155.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-06-23T18:33:02.744 INFO:teuthology.orchestra.run.smithi155.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-06-23T18:33:02.769 INFO:teuthology.orchestra.run.smithi155.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-06-23T18:33:02.795 INFO:teuthology.orchestra.run.smithi155.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-06-23T18:33:02.813 INFO:teuthology.orchestra.run.smithi079.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 57 MB/s | 47 MB 00:00 2023-06-23T18:33:02.820 INFO:teuthology.orchestra.run.smithi155.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-06-23T18:33:02.870 INFO:teuthology.orchestra.run.smithi155.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 513 kB/s | 39 kB 00:00 2023-06-23T18:33:02.904 INFO:teuthology.orchestra.run.smithi155.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 7.5 MB/s | 644 kB 00:00 2023-06-23T18:33:02.929 INFO:teuthology.orchestra.run.smithi155.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-06-23T18:33:02.954 INFO:teuthology.orchestra.run.smithi155.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-06-23T18:33:03.004 INFO:teuthology.orchestra.run.smithi155.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.4 MB/s | 2.3 MB 00:00 2023-06-23T18:33:03.030 INFO:teuthology.orchestra.run.smithi155.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2023-06-23T18:33:03.055 INFO:teuthology.orchestra.run.smithi155.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-06-23T18:33:03.080 INFO:teuthology.orchestra.run.smithi155.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-06-23T18:33:03.114 INFO:teuthology.orchestra.run.smithi155.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-06-23T18:33:03.148 INFO:teuthology.orchestra.run.smithi155.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 580 kB/s | 39 kB 00:00 2023-06-23T18:33:03.190 INFO:teuthology.orchestra.run.smithi155.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 8.9 MB/s | 688 kB 00:00 2023-06-23T18:33:03.240 INFO:teuthology.orchestra.run.smithi155.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 5.7 MB/s | 538 kB 00:00 2023-06-23T18:33:03.265 INFO:teuthology.orchestra.run.smithi155.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.0 MB/s | 157 kB 00:00 2023-06-23T18:33:03.290 INFO:teuthology.orchestra.run.smithi155.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-06-23T18:33:03.332 INFO:teuthology.orchestra.run.smithi155.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-06-23T18:33:03.374 INFO:teuthology.orchestra.run.smithi155.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 638 kB/s | 27 kB 00:00 2023-06-23T18:33:03.424 INFO:teuthology.orchestra.run.smithi155.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-06-23T18:33:03.466 INFO:teuthology.orchestra.run.smithi155.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-06-23T18:33:03.533 INFO:teuthology.orchestra.run.smithi155.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-06-23T18:33:03.600 INFO:teuthology.orchestra.run.smithi155.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 6.7 MB/s | 457 kB 00:00 2023-06-23T18:33:03.642 INFO:teuthology.orchestra.run.smithi155.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-06-23T18:33:03.709 INFO:teuthology.orchestra.run.smithi155.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-06-23T18:33:03.734 INFO:teuthology.orchestra.run.smithi155.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 697 kB/s | 64 kB 00:00 2023-06-23T18:33:03.759 INFO:teuthology.orchestra.run.smithi155.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-06-23T18:33:03.785 INFO:teuthology.orchestra.run.smithi155.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-06-23T18:33:03.810 INFO:teuthology.orchestra.run.smithi155.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-06-23T18:33:03.843 INFO:teuthology.orchestra.run.smithi155.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-06-23T18:33:04.002 INFO:teuthology.orchestra.run.smithi155.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-06-23T18:33:04.027 INFO:teuthology.orchestra.run.smithi155.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2023-06-23T18:33:04.069 INFO:teuthology.orchestra.run.smithi155.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-06-23T18:33:04.128 INFO:teuthology.orchestra.run.smithi155.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2023-06-23T18:33:04.212 INFO:teuthology.orchestra.run.smithi155.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-06-23T18:33:04.279 INFO:teuthology.orchestra.run.smithi155.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-06-23T18:33:04.320 INFO:teuthology.orchestra.run.smithi155.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-06-23T18:33:04.388 INFO:teuthology.orchestra.run.smithi155.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-06-23T18:33:04.471 INFO:teuthology.orchestra.run.smithi155.stdout:(128/131): postgresql-server-10.21-2.module+el8 15 MB/s | 5.1 MB 00:00 2023-06-23T18:33:04.530 INFO:teuthology.orchestra.run.smithi155.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-06-23T18:33:04.732 INFO:teuthology.orchestra.run.smithi155.stdout:(130/131): librados2-18.0.0-4479.g8627af3e.el8. 8.9 MB/s | 3.7 MB 00:00 2023-06-23T18:33:04.830 INFO:teuthology.orchestra.run.smithi155.stdout:(131/131): librbd1-18.0.0-4479.g8627af3e.el8.x8 8.0 MB/s | 3.6 MB 00:00 2023-06-23T18:33:04.834 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2023-06-23T18:33:04.834 INFO:teuthology.orchestra.run.smithi155.stdout:Total 22 MB/s | 205 MB 00:09 2023-06-23T18:33:05.649 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2023-06-23T18:33:05.769 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2023-06-23T18:33:05.769 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2023-06-23T18:33:06.880 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2023-06-23T18:33:06.884 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2023-06-23T18:33:12.031 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2023-06-23T18:33:12.202 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-06-23T18:33:12.410 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-06-23T18:33:12.587 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-23T18:33:12.734 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-23T18:33:12.937 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-23T18:33:13.367 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-23T18:33:13.596 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 5/134 2023-06-23T18:33:13.707 INFO:teuthology.orchestra.run.smithi079.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T18:33:13.708 INFO:teuthology.orchestra.run.smithi079.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T18:33:13.721 INFO:teuthology.orchestra.run.smithi079.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-06-23T18:33:13.723 INFO:teuthology.orchestra.run.smithi079.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-06-23T18:33:13.724 INFO:teuthology.orchestra.run.smithi079.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-06-23T18:33:13.725 INFO:teuthology.orchestra.run.smithi079.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-23T18:33:13.727 INFO:teuthology.orchestra.run.smithi079.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-23T18:33:13.730 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-23T18:33:13.731 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-23T18:33:13.860 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-06-23T18:33:13.872 INFO:teuthology.orchestra.run.smithi079.stdout:=========================================================================================================================== 2023-06-23T18:33:13.872 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2023-06-23T18:33:13.872 INFO:teuthology.orchestra.run.smithi079.stdout:=========================================================================================================================== 2023-06-23T18:33:13.873 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-06-23T18:33:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-06-23T18:33:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-23T18:33:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: ceph x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 6.4 k 2023-06-23T18:33:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 5.4 M 2023-06-23T18:33:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 848 k 2023-06-23T18:33:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 165 k 2023-06-23T18:33:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 1.5 M 2023-06-23T18:33:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 128 k 2023-06-23T18:33:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 1.7 M 2023-06-23T18:33:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 7.4 M 2023-06-23T18:33:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 47 k 2023-06-23T18:33:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 13 M 2023-06-23T18:33:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 48 M 2023-06-23T18:33:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 258 k 2023-06-23T18:33:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 161 k 2023-06-23T18:33:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 258 k 2023-06-23T18:33:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 18 k 2023-06-23T18:33:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-06-23T18:33:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-23T18:33:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-23T18:33:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 31 k 2023-06-23T18:33:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 771 k 2023-06-23T18:33:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 139 k 2023-06-23T18:33:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-23T18:33:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-06-23T18:33:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-23T18:33:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-06-23T18:33:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-06-23T18:33:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-06-23T18:33:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 197 k 2023-06-23T18:33:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 370 k 2023-06-23T18:33:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 365 k 2023-06-23T18:33:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 107 k 2023-06-23T18:33:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 94 k 2023-06-23T18:33:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.4 M 2023-06-23T18:33:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 179 k 2023-06-23T18:33:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-06-23T18:33:13.878 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2023-06-23T18:33:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-06-23T18:33:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: librados2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.7 M 2023-06-23T18:33:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.6 M 2023-06-23T18:33:13.878 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2023-06-23T18:33:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-23T18:33:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 22 M 2023-06-23T18:33:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 23 k 2023-06-23T18:33:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 2.3 M 2023-06-23T18:33:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 240 k 2023-06-23T18:33:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 4.6 M 2023-06-23T18:33:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 18 M 2023-06-23T18:33:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 15 k 2023-06-23T18:33:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 24 k 2023-06-23T18:33:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-23T18:33:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-23T18:33:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-06-23T18:33:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-23T18:33:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-06-23T18:33:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 137 k 2023-06-23T18:33:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-06-23T18:33:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-06-23T18:33:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-23T18:33:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-06-23T18:33:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-06-23T18:33:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-23T18:33:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 474 k 2023-06-23T18:33:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-06-23T18:33:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 6.7 M 2023-06-23T18:33:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-23T18:33:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-06-23T18:33:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-23T18:33:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-23T18:33:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-23T18:33:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-23T18:33:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-23T18:33:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-06-23T18:33:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-23T18:33:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-23T18:33:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-23T18:33:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-23T18:33:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-23T18:33:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-23T18:33:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-23T18:33:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-23T18:33:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 45 k 2023-06-23T18:33:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 119 k 2023-06-23T18:33:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-23T18:33:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-06-23T18:33:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-23T18:33:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-23T18:33:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-23T18:33:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-23T18:33:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-23T18:33:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-23T18:33:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-23T18:33:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-23T18:33:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-23T18:33:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T18:33:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-23T18:33:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-23T18:33:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-23T18:33:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-23T18:33:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-23T18:33:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-23T18:33:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-23T18:33:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-23T18:33:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-23T18:33:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-23T18:33:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-23T18:33:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-23T18:33:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-06-23T18:33:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-23T18:33:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-23T18:33:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-06-23T18:33:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-23T18:33:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-23T18:33:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T18:33:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-23T18:33:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-23T18:33:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-06-23T18:33:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-23T18:33:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-23T18:33:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-23T18:33:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-23T18:33:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-23T18:33:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-23T18:33:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-23T18:33:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-23T18:33:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-23T18:33:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-23T18:33:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-06-23T18:33:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-23T18:33:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-06-23T18:33:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-23T18:33:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-23T18:33:13.891 INFO:teuthology.orchestra.run.smithi079.stdout:Enabling module streams: 2023-06-23T18:33:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql 10 2023-06-23T18:33:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:33:13.891 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-06-23T18:33:13.891 INFO:teuthology.orchestra.run.smithi079.stdout:=========================================================================================================================== 2023-06-23T18:33:13.891 INFO:teuthology.orchestra.run.smithi079.stdout:Install 128 Packages 2023-06-23T18:33:13.891 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 3 Packages 2023-06-23T18:33:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:33:13.892 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 205 M 2023-06-23T18:33:13.892 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-06-23T18:33:13.987 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 5/134 2023-06-23T18:33:14.203 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:33:14.212 INFO:teuthology.orchestra.run.smithi079.stdout:(1/131): ceph-18.0.0-4479.g8627af3e.el8.x86_64. 46 kB/s | 6.4 kB 00:00 2023-06-23T18:33:14.321 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:33:14.504 INFO:teuthology.orchestra.run.smithi079.stdout:(2/131): ceph-fuse-18.0.0-4479.g8627af3e.el8.x8 2.8 MB/s | 848 kB 00:00 2023-06-23T18:33:14.554 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:33:14.604 INFO:teuthology.orchestra.run.smithi079.stdout:(3/131): ceph-immutable-object-cache-18.0.0-447 1.6 MB/s | 165 kB 00:00 2023-06-23T18:33:14.634 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:33:14.729 INFO:teuthology.orchestra.run.smithi079.stdout:(4/131): ceph-base-18.0.0-4479.g8627af3e.el8.x8 8.2 MB/s | 5.4 MB 00:00 2023-06-23T18:33:14.817 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 8/134 2023-06-23T18:33:14.913 INFO:teuthology.orchestra.run.smithi079.stdout:(5/131): ceph-mgr-18.0.0-4479.g8627af3e.el8.x86 8.4 MB/s | 1.5 MB 00:00 2023-06-23T18:33:15.046 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-06-23T18:33:15.213 INFO:teuthology.orchestra.run.smithi079.stdout:(6/131): ceph-mds-18.0.0-4479.g8627af3e.el8.x86 3.8 MB/s | 2.3 MB 00:00 2023-06-23T18:33:15.286 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-06-23T18:33:15.405 INFO:teuthology.orchestra.run.smithi079.stdout:(7/131): ceph-mon-18.0.0-4479.g8627af3e.el8.x86 9.4 MB/s | 4.6 MB 00:00 2023-06-23T18:33:15.734 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-06-23T18:33:15.972 INFO:teuthology.orchestra.run.smithi079.stdout:(8/131): ceph-common-18.0.0-4479.g8627af3e.el8. 12 MB/s | 22 MB 00:01 2023-06-23T18:33:16.039 INFO:teuthology.orchestra.run.smithi079.stdout:(9/131): ceph-selinux-18.0.0-4479.g8627af3e.el8 365 kB/s | 24 kB 00:00 2023-06-23T18:33:16.167 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-06-23T18:33:16.441 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-06-23T18:33:16.612 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 14/134 2023-06-23T18:33:16.715 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 14/134 2023-06-23T18:33:16.864 INFO:teuthology.orchestra.run.smithi079.stdout:(10/131): ceph-radosgw-18.0.0-4479.g8627af3e.el 9.2 MB/s | 13 MB 00:01 2023-06-23T18:33:16.903 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 15/134 2023-06-23T18:33:16.943 INFO:teuthology.orchestra.run.smithi079.stdout:(11/131): cephfs-mirror-18.0.0-4479.g8627af3e.e 3.4 MB/s | 258 kB 00:00 2023-06-23T18:33:17.006 INFO:teuthology.orchestra.run.smithi079.stdout:(12/131): libcephfs-devel-18.0.0-4479.g8627af3e 467 kB/s | 31 kB 00:00 2023-06-23T18:33:17.036 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 15/134 2023-06-23T18:33:17.156 INFO:teuthology.orchestra.run.smithi079.stdout:(13/131): libcephfs2-18.0.0-4479.g8627af3e.el8. 5.0 MB/s | 771 kB 00:00 2023-06-23T18:33:17.248 INFO:teuthology.orchestra.run.smithi079.stdout:(14/131): libcephsqlite-18.0.0-4479.g8627af3e.e 1.5 MB/s | 137 kB 00:00 2023-06-23T18:33:17.266 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-06-23T18:33:17.451 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-06-23T18:33:17.466 INFO:teuthology.orchestra.run.smithi079.stdout:(15/131): ceph-osd-18.0.0-4479.g8627af3e.el8.x8 7.9 MB/s | 18 MB 00:02 2023-06-23T18:33:17.491 INFO:teuthology.orchestra.run.smithi079.stdout:(16/131): librados-devel-18.0.0-4479.g8627af3e. 575 kB/s | 139 kB 00:00 2023-06-23T18:33:17.524 INFO:teuthology.orchestra.run.smithi079.stdout:(17/131): libradosstriper1-18.0.0-4479.g8627af3 7.9 MB/s | 474 kB 00:00 2023-06-23T18:33:17.591 INFO:teuthology.orchestra.run.smithi079.stdout:(18/131): python3-ceph-argparse-18.0.0-4479.g86 673 kB/s | 45 kB 00:00 2023-06-23T18:33:17.643 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-06-23T18:33:17.667 INFO:teuthology.orchestra.run.smithi079.stdout:(19/131): python3-ceph-common-18.0.0-4479.g8627 1.5 MB/s | 119 kB 00:00 2023-06-23T18:33:17.742 INFO:teuthology.orchestra.run.smithi079.stdout:(20/131): python3-cephfs-18.0.0-4479.g8627af3e. 2.6 MB/s | 197 kB 00:00 2023-06-23T18:33:17.769 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-23T18:33:17.825 INFO:teuthology.orchestra.run.smithi079.stdout:(21/131): python3-rados-18.0.0-4479.g8627af3e.e 4.4 MB/s | 370 kB 00:00 2023-06-23T18:33:17.909 INFO:teuthology.orchestra.run.smithi079.stdout:(22/131): python3-rbd-18.0.0-4479.g8627af3e.el8 4.3 MB/s | 365 kB 00:00 2023-06-23T18:33:17.922 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-23T18:33:17.968 INFO:teuthology.orchestra.run.smithi079.stdout:(23/131): python3-rgw-18.0.0-4479.g8627af3e.el8 1.8 MB/s | 107 kB 00:00 2023-06-23T18:33:18.035 INFO:teuthology.orchestra.run.smithi079.stdout:(24/131): rbd-fuse-18.0.0-4479.g8627af3e.el8.x8 1.4 MB/s | 94 kB 00:00 2023-06-23T18:33:18.078 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-23T18:33:18.260 INFO:teuthology.orchestra.run.smithi079.stdout:(25/131): librgw2-18.0.0-4479.g8627af3e.el8.x86 8.8 MB/s | 6.7 MB 00:00 2023-06-23T18:33:18.343 INFO:teuthology.orchestra.run.smithi079.stdout:(26/131): rbd-nbd-18.0.0-4479.g8627af3e.el8.x86 2.1 MB/s | 179 kB 00:00 2023-06-23T18:33:18.444 INFO:teuthology.orchestra.run.smithi079.stdout:(27/131): rbd-mirror-18.0.0-4479.g8627af3e.el8. 8.4 MB/s | 3.4 MB 00:00 2023-06-23T18:33:18.461 INFO:teuthology.orchestra.run.smithi079.stdout:(28/131): ceph-grafana-dashboards-18.0.0-4479.g 197 kB/s | 23 kB 00:00 2023-06-23T18:33:18.495 INFO:teuthology.orchestra.run.smithi079.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4479.g8627af3 2.5 MB/s | 128 kB 00:00 2023-06-23T18:33:18.691 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-23T18:33:18.736 INFO:teuthology.orchestra.run.smithi079.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4479.g8627a 6.2 MB/s | 1.7 MB 00:00 2023-06-23T18:33:18.820 INFO:teuthology.orchestra.run.smithi079.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4479.g86 2.8 MB/s | 240 kB 00:00 2023-06-23T18:33:18.886 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-23T18:33:18.887 INFO:teuthology.orchestra.run.smithi079.stdout:(32/131): ceph-mgr-rook-18.0.0-4479.g8627af3e.e 713 kB/s | 47 kB 00:00 2023-06-23T18:33:18.962 INFO:teuthology.orchestra.run.smithi079.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4479.g8 193 kB/s | 15 kB 00:00 2023-06-23T18:33:19.046 INFO:teuthology.orchestra.run.smithi079.stdout:(34/131): ceph-volume-18.0.0-4479.g8627af3e.el8 3.0 MB/s | 258 kB 00:00 2023-06-23T18:33:19.121 INFO:teuthology.orchestra.run.smithi079.stdout:(35/131): cephadm-18.0.0-4479.g8627af3e.el8.noa 2.1 MB/s | 161 kB 00:00 2023-06-23T18:33:19.272 INFO:teuthology.orchestra.run.smithi079.stdout:(36/131): ceph-mgr-diskprediction-local-18.0.0- 9.5 MB/s | 7.4 MB 00:00 2023-06-23T18:33:19.288 INFO:teuthology.orchestra.run.smithi079.stdout:(37/131): cephfs-top-18.0.0-4479.g8627af3e.el8. 108 kB/s | 18 kB 00:00 2023-06-23T18:33:19.390 INFO:teuthology.orchestra.run.smithi079.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 403 for https://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 152.19.134.145) 2023-06-23T18:33:19.394 INFO:teuthology.orchestra.run.smithi079.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 403 for https://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 152.19.134.145) 2023-06-23T18:33:19.482 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-23T18:33:19.669 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-06-23T18:33:19.872 INFO:teuthology.orchestra.run.smithi079.stdout:(38/131): ceph-test-18.0.0-4479.g8627af3e.el8.x 12 MB/s | 48 MB 00:03 2023-06-23T18:33:19.900 INFO:teuthology.orchestra.run.smithi079.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 403 for http://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 152.19.134.145) 2023-06-23T18:33:19.901 INFO:teuthology.orchestra.run.smithi079.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 403 for http://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 152.19.134.145) 2023-06-23T18:33:19.901 INFO:teuthology.orchestra.run.smithi079.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 403 for https://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 152.19.134.145) 2023-06-23T18:33:19.929 INFO:teuthology.orchestra.run.smithi079.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 403 for http://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 152.19.134.145) 2023-06-23T18:33:20.072 INFO:teuthology.orchestra.run.smithi079.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 75 kB/s | 59 kB 00:00 2023-06-23T18:33:20.106 INFO:teuthology.orchestra.run.smithi079.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 322 kB/s | 75 kB 00:00 2023-06-23T18:33:20.107 INFO:teuthology.orchestra.run.smithi079.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Status code: 403 for https://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm (IP: 152.19.134.145) 2023-06-23T18:33:20.148 INFO:teuthology.orchestra.run.smithi079.stdout:(41/131): gperftools-libs-2.7-9.el8.x86_64.rpm 350 kB/s | 306 kB 00:00 2023-06-23T18:33:20.149 INFO:teuthology.orchestra.run.smithi079.stdout:[MIRROR] python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm: Status code: 403 for https://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm (IP: 152.19.134.145) 2023-06-23T18:33:20.150 INFO:teuthology.orchestra.run.smithi079.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Status code: 403 for http://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm (IP: 152.19.134.145) 2023-06-23T18:33:20.216 INFO:teuthology.orchestra.run.smithi079.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2023-06-23T18:33:20.240 INFO:teuthology.orchestra.run.smithi079.stdout:(43/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 326 kB/s | 44 kB 00:00 2023-06-23T18:33:20.274 INFO:teuthology.orchestra.run.smithi079.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 565 kB/s | 33 kB 00:00 2023-06-23T18:33:20.299 INFO:teuthology.orchestra.run.smithi079.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 279 kB/s | 16 kB 00:00 2023-06-23T18:33:20.324 INFO:teuthology.orchestra.run.smithi079.stdout:(46/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.8 MB/s | 476 kB 00:00 2023-06-23T18:33:20.350 INFO:teuthology.orchestra.run.smithi079.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-06-23T18:33:20.383 INFO:teuthology.orchestra.run.smithi079.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-06-23T18:33:20.402 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-06-23T18:33:20.409 INFO:teuthology.orchestra.run.smithi079.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-06-23T18:33:20.442 INFO:teuthology.orchestra.run.smithi079.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 300 kB/s | 18 kB 00:00 2023-06-23T18:33:20.467 INFO:teuthology.orchestra.run.smithi079.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.2 MB/s | 384 kB 00:00 2023-06-23T18:33:20.501 INFO:teuthology.orchestra.run.smithi079.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 833 kB/s | 49 kB 00:00 2023-06-23T18:33:20.534 INFO:teuthology.orchestra.run.smithi079.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 879 kB/s | 59 kB 00:00 2023-06-23T18:33:20.577 INFO:teuthology.orchestra.run.smithi079.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 800 kB/s | 60 kB 00:00 2023-06-23T18:33:20.610 INFO:teuthology.orchestra.run.smithi079.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 5.8 MB/s | 1.2 MB 00:00 2023-06-23T18:33:20.664 INFO:teuthology.orchestra.run.smithi079.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2023-06-23T18:33:20.689 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-06-23T18:33:20.694 INFO:teuthology.orchestra.run.smithi079.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 133 kB/s | 16 kB 00:00 2023-06-23T18:33:20.719 INFO:teuthology.orchestra.run.smithi079.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 309 kB/s | 34 kB 00:00 2023-06-23T18:33:20.744 INFO:teuthology.orchestra.run.smithi079.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.4 MB/s | 196 kB 00:00 2023-06-23T18:33:20.770 INFO:teuthology.orchestra.run.smithi079.stdout:(60/131): python3-rsa-4.8-1.el8.noarch.rpm 814 kB/s | 61 kB 00:00 2023-06-23T18:33:20.795 INFO:teuthology.orchestra.run.smithi079.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2023-06-23T18:33:20.820 INFO:teuthology.orchestra.run.smithi079.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2023-06-23T18:33:20.846 INFO:teuthology.orchestra.run.smithi079.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 386 kB/s | 29 kB 00:00 2023-06-23T18:33:20.871 INFO:teuthology.orchestra.run.smithi079.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 364 kB/s | 27 kB 00:00 2023-06-23T18:33:20.902 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-06-23T18:33:20.921 INFO:teuthology.orchestra.run.smithi079.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-06-23T18:33:20.947 INFO:teuthology.orchestra.run.smithi079.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 813 kB/s | 61 kB 00:00 2023-06-23T18:33:20.972 INFO:teuthology.orchestra.run.smithi079.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.0 MB/s | 251 kB 00:00 2023-06-23T18:33:20.997 INFO:teuthology.orchestra.run.smithi079.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-06-23T18:33:21.022 INFO:teuthology.orchestra.run.smithi079.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 303 kB/s | 23 kB 00:00 2023-06-23T18:33:21.048 INFO:teuthology.orchestra.run.smithi079.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-06-23T18:33:21.098 INFO:teuthology.orchestra.run.smithi079.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 624 kB/s | 47 kB 00:00 2023-06-23T18:33:21.123 INFO:teuthology.orchestra.run.smithi079.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.8 MB/s | 200 kB 00:00 2023-06-23T18:33:21.148 INFO:teuthology.orchestra.run.smithi079.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 853 kB/s | 43 kB 00:00 2023-06-23T18:33:21.174 INFO:teuthology.orchestra.run.smithi079.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-06-23T18:33:21.199 INFO:teuthology.orchestra.run.smithi079.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 957 kB/s | 48 kB 00:00 2023-06-23T18:33:21.224 INFO:teuthology.orchestra.run.smithi079.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-06-23T18:33:21.258 INFO:teuthology.orchestra.run.smithi079.stdout:(77/131): thrift-0.13.0-2.el8.x86_64.rpm 6.1 MB/s | 1.7 MB 00:00 2023-06-23T18:33:21.283 INFO:teuthology.orchestra.run.smithi079.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 510 kB/s | 43 kB 00:00 2023-06-23T18:33:21.317 INFO:teuthology.orchestra.run.smithi079.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.2 MB/s | 109 kB 00:00 2023-06-23T18:33:21.342 INFO:teuthology.orchestra.run.smithi079.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2023-06-23T18:33:21.367 INFO:teuthology.orchestra.run.smithi079.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 372 kB/s | 19 kB 00:00 2023-06-23T18:33:21.393 INFO:teuthology.orchestra.run.smithi079.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-06-23T18:33:21.418 INFO:teuthology.orchestra.run.smithi079.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.2 MB/s | 560 kB 00:00 2023-06-23T18:33:21.443 INFO:teuthology.orchestra.run.smithi079.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 358 kB/s | 18 kB 00:00 2023-06-23T18:33:21.477 INFO:teuthology.orchestra.run.smithi079.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 5.0 MB/s | 559 kB 00:00 2023-06-23T18:33:21.495 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-06-23T18:33:21.511 INFO:teuthology.orchestra.run.smithi079.stdout:(86/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2023-06-23T18:33:21.536 INFO:teuthology.orchestra.run.smithi079.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-06-23T18:33:21.561 INFO:teuthology.orchestra.run.smithi079.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-06-23T18:33:21.586 INFO:teuthology.orchestra.run.smithi079.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-06-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi079.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 893 kB/s | 60 kB 00:00 2023-06-23T18:33:21.653 INFO:teuthology.orchestra.run.smithi079.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.4 MB/s | 98 kB 00:00 2023-06-23T18:33:21.679 INFO:teuthology.orchestra.run.smithi079.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-06-23T18:33:21.702 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:18.0.0-4479.g8627af3e.el8. 26/134 2023-06-23T18:33:21.703 INFO:teuthology.orchestra.run.smithi079.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 769 kB/s | 39 kB 00:00 2023-06-23T18:33:21.746 INFO:teuthology.orchestra.run.smithi079.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-06-23T18:33:21.771 INFO:teuthology.orchestra.run.smithi079.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-06-23T18:33:21.796 INFO:teuthology.orchestra.run.smithi079.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-06-23T18:33:21.823 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el 27/134 2023-06-23T18:33:21.855 INFO:teuthology.orchestra.run.smithi079.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.1 MB/s | 2.3 MB 00:00 2023-06-23T18:33:21.880 INFO:teuthology.orchestra.run.smithi079.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-06-23T18:33:21.898 INFO:teuthology.orchestra.run.smithi079.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-06-23T18:33:21.923 INFO:teuthology.orchestra.run.smithi079.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-06-23T18:33:21.956 INFO:teuthology.orchestra.run.smithi079.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-06-23T18:33:21.981 INFO:teuthology.orchestra.run.smithi079.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 664 kB/s | 39 kB 00:00 2023-06-23T18:33:22.015 INFO:teuthology.orchestra.run.smithi079.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-06-23T18:33:22.049 INFO:teuthology.orchestra.run.smithi079.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.9 MB/s | 538 kB 00:00 2023-06-23T18:33:22.067 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_6 28/134 2023-06-23T18:33:22.074 INFO:teuthology.orchestra.run.smithi079.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-06-23T18:33:22.099 INFO:teuthology.orchestra.run.smithi079.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-06-23T18:33:22.141 INFO:teuthology.orchestra.run.smithi079.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-06-23T18:33:22.183 INFO:teuthology.orchestra.run.smithi079.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 639 kB/s | 27 kB 00:00 2023-06-23T18:33:22.225 INFO:teuthology.orchestra.run.smithi079.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-06-23T18:33:22.253 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-23T18:33:22.267 INFO:teuthology.orchestra.run.smithi079.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-06-23T18:33:22.359 INFO:teuthology.orchestra.run.smithi079.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-06-23T18:33:22.409 INFO:teuthology.orchestra.run.smithi079.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-06-23T18:33:22.451 INFO:teuthology.orchestra.run.smithi079.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-06-23T18:33:22.518 INFO:teuthology.orchestra.run.smithi079.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-06-23T18:33:22.543 INFO:teuthology.orchestra.run.smithi079.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 696 kB/s | 64 kB 00:00 2023-06-23T18:33:22.602 INFO:teuthology.orchestra.run.smithi079.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.5 MB/s | 126 kB 00:00 2023-06-23T18:33:22.636 INFO:teuthology.orchestra.run.smithi079.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 218 kB/s | 20 kB 00:00 2023-06-23T18:33:22.661 INFO:teuthology.orchestra.run.smithi079.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.7 MB/s | 103 kB 00:00 2023-06-23T18:33:22.702 INFO:teuthology.orchestra.run.smithi079.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-06-23T18:33:22.861 INFO:teuthology.orchestra.run.smithi079.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-06-23T18:33:22.886 INFO:teuthology.orchestra.run.smithi079.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2023-06-23T18:33:22.929 INFO:teuthology.orchestra.run.smithi079.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-06-23T18:33:22.979 INFO:teuthology.orchestra.run.smithi079.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-06-23T18:33:23.079 INFO:teuthology.orchestra.run.smithi079.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-06-23T18:33:23.146 INFO:teuthology.orchestra.run.smithi079.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-06-23T18:33:23.179 INFO:teuthology.orchestra.run.smithi079.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.9 MB/s | 808 kB 00:00 2023-06-23T18:33:23.238 INFO:teuthology.orchestra.run.smithi079.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-06-23T18:33:23.314 INFO:teuthology.orchestra.run.smithi079.stdout:(128/131): postgresql-server-10.21-2.module+el8 15 MB/s | 5.1 MB 00:00 2023-06-23T18:33:23.372 INFO:teuthology.orchestra.run.smithi079.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 15 MB/s | 798 kB 00:00 2023-06-23T18:33:24.054 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-23T18:33:24.214 INFO:teuthology.orchestra.run.smithi079.stdout:(130/131): librados2-18.0.0-4479.g8627af3e.el8. 3.5 MB/s | 3.7 MB 00:01 2023-06-23T18:33:24.864 INFO:teuthology.orchestra.run.smithi079.stdout:(131/131): librbd1-18.0.0-4479.g8627af3e.el8.x8 2.2 MB/s | 3.6 MB 00:01 2023-06-23T18:33:24.868 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-06-23T18:33:24.868 INFO:teuthology.orchestra.run.smithi079.stdout:Total 19 MB/s | 205 MB 00:10 2023-06-23T18:33:25.127 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-06-23T18:33:25.436 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-06-23T18:33:25.700 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-06-23T18:33:25.731 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-06-23T18:33:25.820 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-06-23T18:33:25.820 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-06-23T18:33:25.958 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-06-23T18:33:26.173 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-06-23T18:33:26.355 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-06-23T18:33:26.747 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 36/134 2023-06-23T18:33:26.938 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-06-23T18:33:26.942 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-06-23T18:33:27.134 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_6 37/134 2023-06-23T18:33:27.318 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-06-23T18:33:27.462 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-06-23T18:33:27.693 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-06-23T18:33:27.749 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-06-23T18:33:27.972 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-06-23T18:33:28.214 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-06-23T18:33:28.418 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-23T18:33:28.502 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-23T18:33:28.708 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-06-23T18:33:28.906 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-06-23T18:33:29.080 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-06-23T18:33:29.296 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-06-23T18:33:29.471 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-06-23T18:33:29.639 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-06-23T18:33:29.828 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-06-23T18:33:30.059 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-06-23T18:33:30.409 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-06-23T18:33:30.745 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-06-23T18:33:30.954 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-06-23T18:33:31.241 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-06-23T18:33:31.481 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-06-23T18:33:31.820 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-06-23T18:33:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-06-23T18:33:32.034 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-06-23T18:33:32.112 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-06-23T18:33:32.278 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-06-23T18:33:32.329 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-06-23T18:33:32.489 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-23T18:33:32.506 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-06-23T18:33:32.619 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-23T18:33:32.725 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-06-23T18:33:32.814 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-23T18:33:32.924 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-06-23T18:33:33.217 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-23T18:33:33.238 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-06-23T18:33:33.480 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 5/134 2023-06-23T18:33:33.517 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-06-23T18:33:33.790 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-06-23T18:33:33.863 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 5/134 2023-06-23T18:33:34.052 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-06-23T18:33:34.072 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:33:34.180 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:33:34.217 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-06-23T18:33:34.422 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:33:34.429 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-06-23T18:33:34.518 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:33:34.689 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-06-23T18:33:34.702 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 8/134 2023-06-23T18:33:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-06-23T18:33:34.924 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-06-23T18:33:35.088 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-06-23T18:33:35.142 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-06-23T18:33:35.278 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-06-23T18:33:35.358 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-06-23T18:33:35.486 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-06-23T18:33:35.723 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-06-23T18:33:35.762 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-06-23T18:33:35.955 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-06-23T18:33:36.026 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-06-23T18:33:36.112 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-06-23T18:33:36.239 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 14/134 2023-06-23T18:33:36.257 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-06-23T18:33:36.341 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 14/134 2023-06-23T18:33:36.500 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-06-23T18:33:36.505 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 15/134 2023-06-23T18:33:36.651 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 15/134 2023-06-23T18:33:36.730 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-06-23T18:33:36.842 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-06-23T18:33:36.923 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-06-23T18:33:37.027 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-06-23T18:33:37.210 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-06-23T18:33:37.226 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-06-23T18:33:37.454 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-23T18:33:37.517 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-06-23T18:33:37.581 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-23T18:33:37.666 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-06-23T18:33:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-23T18:33:37.889 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-06-23T18:33:38.358 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-23T18:33:38.521 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-23T18:33:38.989 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-06-23T18:33:39.114 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-23T18:33:39.238 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-06-23T18:33:39.346 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-06-23T18:33:39.500 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-06-23T18:33:39.839 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-06-23T18:33:40.045 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-06-23T18:33:40.135 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-06-23T18:33:40.338 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-06-23T18:33:40.341 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-06-23T18:33:40.528 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-06-23T18:33:40.764 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-06-23T18:33:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-common-2:18.0.0-4479.g8627af3e.el8. 26/134 2023-06-23T18:33:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el 27/134 2023-06-23T18:33:41.148 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-06-23T18:33:41.333 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 92/134 2023-06-23T18:33:41.353 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_6 28/134 2023-06-23T18:33:41.405 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 92/134 2023-06-23T18:33:41.570 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-23T18:33:41.625 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 93/134 2023-06-23T18:33:43.339 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-23T18:33:44.357 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-06-23T18:33:44.675 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-06-23T18:33:44.974 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-06-23T18:33:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-06-23T18:33:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-06-23T18:33:45.430 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:33:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-06-23T18:33:45.666 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:33:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 36/134 2023-06-23T18:33:45.967 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_6 37/134 2023-06-23T18:33:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-06-23T18:33:46.263 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:33:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-06-23T18:33:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-06-23T18:33:46.561 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 95/134 2023-06-23T18:33:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-06-23T18:33:46.632 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 95/134 2023-06-23T18:33:46.772 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-06-23T18:33:46.816 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 96/134 2023-06-23T18:33:47.014 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-06-23T18:33:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-23T18:33:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-23T18:33:47.481 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-06-23T18:33:47.639 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-06-23T18:33:47.828 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-06-23T18:33:48.053 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-06-23T18:33:48.254 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-06-23T18:33:48.397 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-06-23T18:33:48.591 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-06-23T18:33:48.772 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-06-23T18:33:49.135 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-06-23T18:33:49.445 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-06-23T18:33:49.672 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-06-23T18:33:49.915 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-06-23T18:33:50.165 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-06-23T18:33:50.493 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-06-23T18:33:50.726 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-06-23T18:33:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-06-23T18:33:51.155 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-06-23T18:33:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-06-23T18:33:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-06-23T18:33:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-06-23T18:33:52.098 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-06-23T18:33:52.339 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-06-23T18:33:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-06-23T18:33:52.751 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-06-23T18:33:52.929 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-06-23T18:33:53.236 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-06-23T18:33:53.449 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-06-23T18:33:53.589 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-06-23T18:33:53.745 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-06-23T18:33:53.926 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-06-23T18:33:54.154 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-06-23T18:33:54.336 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-06-23T18:33:54.495 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-06-23T18:33:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-06-23T18:33:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-06-23T18:33:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-06-23T18:33:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-06-23T18:33:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-06-23T18:33:55.759 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-06-23T18:33:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-06-23T18:33:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-06-23T18:33:57.262 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-06-23T18:33:57.495 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-06-23T18:33:57.750 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-06-23T18:33:58.035 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-06-23T18:33:58.353 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-06-23T18:33:58.540 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-06-23T18:33:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-06-23T18:33:59.516 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 92/134 2023-06-23T18:33:59.574 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 92/134 2023-06-23T18:33:59.742 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 93/134 2023-06-23T18:34:03.311 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:34:03.875 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:34:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:34:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 95/134 2023-06-23T18:34:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 95/134 2023-06-23T18:34:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 96/134 2023-06-23T18:34:05.741 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 96/134 2023-06-23T18:34:05.741 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2023-06-23T18:34:05.741 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2023-06-23T18:34:05.741 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /dev 2023-06-23T18:34:05.741 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /run 2023-06-23T18:34:05.742 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2023-06-23T18:34:05.742 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2023-06-23T18:34:05.742 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2023-06-23T18:34:05.742 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2023-06-23T18:34:05.742 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /dev 2023-06-23T18:34:05.742 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:34:06.088 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 97/134 2023-06-23T18:34:07.650 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 97/134 2023-06-23T18:34:07.882 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 98/134 2023-06-23T18:34:08.101 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 98/134 2023-06-23T18:34:08.262 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 99/134 2023-06-23T18:34:08.724 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 99/134 2023-06-23T18:34:08.937 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 100/134 2023-06-23T18:34:08.954 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 100/134 2023-06-23T18:34:10.604 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 101/134 2023-06-23T18:34:10.801 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 101/134 2023-06-23T18:34:10.910 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.e 102/134 2023-06-23T18:34:11.068 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e. 103/134 2023-06-23T18:34:11.253 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 104/134 2023-06-23T18:34:11.472 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 104/134 2023-06-23T18:34:11.840 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 105/134 2023-06-23T18:34:12.804 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 105/134 2023-06-23T18:34:12.978 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 106/134 2023-06-23T18:34:13.072 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 106/134 2023-06-23T18:34:13.451 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el 107/134 2023-06-23T18:34:13.640 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 108/134 2023-06-23T18:34:13.676 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 108/134 2023-06-23T18:34:13.861 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 109/134 2023-06-23T18:34:13.872 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 109/134 2023-06-23T18:34:15.156 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 110/134 2023-06-23T18:34:15.391 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 111/134 2023-06-23T18:34:15.428 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 111/134 2023-06-23T18:34:15.611 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4479.g8627a 112/134 2023-06-23T18:34:15.653 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4479.g8627a 112/134 2023-06-23T18:34:15.811 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 113/134 2023-06-23T18:34:16.156 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 113/134 2023-06-23T18:34:16.353 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 114/134 2023-06-23T18:34:20.878 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 114/134 2023-06-23T18:34:21.826 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 115/134 2023-06-23T18:34:22.150 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-06-23T18:34:22.365 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 117/134 2023-06-23T18:34:22.686 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-06-23T18:34:22.851 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-06-23T18:34:23.047 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-06-23T18:34:23.314 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-06-23T18:34:23.489 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_ 121/134 2023-06-23T18:34:23.644 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 96/134 2023-06-23T18:34:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /sys 2023-06-23T18:34:23.645 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /proc 2023-06-23T18:34:23.645 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /dev 2023-06-23T18:34:23.645 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /run 2023-06-23T18:34:23.645 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /mnt 2023-06-23T18:34:23.645 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /var/tmp 2023-06-23T18:34:23.645 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /home 2023-06-23T18:34:23.645 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /tmp 2023-06-23T18:34:23.646 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /dev 2023-06-23T18:34:23.646 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:34:23.661 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-23T18:34:23.986 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 97/134 2023-06-23T18:34:25.544 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 97/134 2023-06-23T18:34:25.770 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 98/134 2023-06-23T18:34:25.811 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:34:25.987 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 98/134 2023-06-23T18:34:26.085 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:34:26.102 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:34:26.127 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 99/134 2023-06-23T18:34:26.293 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 124/134 2023-06-23T18:34:26.567 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 125/134 2023-06-23T18:34:26.588 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 99/134 2023-06-23T18:34:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 100/134 2023-06-23T18:34:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 100/134 2023-06-23T18:34:26.890 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 126/134 2023-06-23T18:34:27.305 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-06-23T18:34:27.526 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-06-23T18:34:27.607 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-06-23T18:34:27.843 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-06-23T18:34:27.855 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-06-23T18:34:27.992 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-23T18:34:28.000 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-23T18:34:28.128 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-06-23T18:34:28.128 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:34:28.362 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:34:28.363 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-23T18:34:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 101/134 2023-06-23T18:34:28.640 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-23T18:34:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 101/134 2023-06-23T18:34:28.774 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.e 102/134 2023-06-23T18:34:28.842 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:34:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e. 103/134 2023-06-23T18:34:29.126 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 104/134 2023-06-23T18:34:29.139 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:34:29.139 INFO:teuthology.orchestra.run.smithi155.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T18:34:29.139 INFO:teuthology.orchestra.run.smithi155.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T18:34:29.139 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:34:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 104/134 2023-06-23T18:34:29.357 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 1/134 2023-06-23T18:34:29.357 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 2/134 2023-06-23T18:34:29.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 3/134 2023-06-23T18:34:29.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 4/134 2023-06-23T18:34:29.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4479.g8627a 5/134 2023-06-23T18:34:29.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:34:29.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:34:29.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 8/134 2023-06-23T18:34:29.358 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 9/134 2023-06-23T18:34:29.359 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 10/134 2023-06-23T18:34:29.359 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 11/134 2023-06-23T18:34:29.359 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 12/134 2023-06-23T18:34:29.359 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 13/134 2023-06-23T18:34:29.359 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_ 14/134 2023-06-23T18:34:29.359 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 15/134 2023-06-23T18:34:29.359 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 16/134 2023-06-23T18:34:29.360 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_6 17/134 2023-06-23T18:34:29.360 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 18/134 2023-06-23T18:34:29.360 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 19/134 2023-06-23T18:34:29.360 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el 20/134 2023-06-23T18:34:29.360 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:18.0.0-4479.g8627af3e.el8. 21/134 2023-06-23T18:34:29.360 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_6 22/134 2023-06-23T18:34:29.360 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 23/134 2023-06-23T18:34:29.360 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 24/134 2023-06-23T18:34:29.367 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 25/134 2023-06-23T18:34:29.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 26/134 2023-06-23T18:34:29.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 27/134 2023-06-23T18:34:29.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 28/134 2023-06-23T18:34:29.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e. 29/134 2023-06-23T18:34:29.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 30/134 2023-06-23T18:34:29.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 31/134 2023-06-23T18:34:29.370 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 32/134 2023-06-23T18:34:29.370 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el 33/134 2023-06-23T18:34:29.371 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 34/134 2023-06-23T18:34:29.371 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.e 35/134 2023-06-23T18:34:29.371 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 36/134 2023-06-23T18:34:29.371 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 37/134 2023-06-23T18:34:29.371 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 38/134 2023-06-23T18:34:29.372 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-06-23T18:34:29.372 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-06-23T18:34:29.372 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-06-23T18:34:29.373 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-06-23T18:34:29.373 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-06-23T18:34:29.374 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-06-23T18:34:29.374 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-06-23T18:34:29.374 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-06-23T18:34:29.375 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-06-23T18:34:29.375 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-06-23T18:34:29.375 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-06-23T18:34:29.376 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-06-23T18:34:29.376 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-06-23T18:34:29.376 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-06-23T18:34:29.378 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-06-23T18:34:29.379 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-06-23T18:34:29.379 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-06-23T18:34:29.379 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-06-23T18:34:29.379 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-06-23T18:34:29.379 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-06-23T18:34:29.379 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-06-23T18:34:29.379 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-06-23T18:34:29.379 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-06-23T18:34:29.380 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-06-23T18:34:29.380 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-06-23T18:34:29.380 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-06-23T18:34:29.380 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-06-23T18:34:29.380 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-06-23T18:34:29.380 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-06-23T18:34:29.381 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-06-23T18:34:29.381 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-06-23T18:34:29.381 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-06-23T18:34:29.381 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-06-23T18:34:29.381 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-06-23T18:34:29.381 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-06-23T18:34:29.381 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-06-23T18:34:29.381 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-06-23T18:34:29.382 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-06-23T18:34:29.382 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-06-23T18:34:29.382 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-06-23T18:34:29.382 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-06-23T18:34:29.382 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-06-23T18:34:29.382 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-06-23T18:34:29.382 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-06-23T18:34:29.382 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-06-23T18:34:29.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-06-23T18:34:29.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-06-23T18:34:29.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-06-23T18:34:29.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-06-23T18:34:29.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-06-23T18:34:29.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-06-23T18:34:29.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-06-23T18:34:29.384 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-06-23T18:34:29.384 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-06-23T18:34:29.384 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-06-23T18:34:29.384 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-06-23T18:34:29.384 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-06-23T18:34:29.384 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-06-23T18:34:29.385 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-06-23T18:34:29.385 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-06-23T18:34:29.385 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-06-23T18:34:29.385 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-06-23T18:34:29.385 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-06-23T18:34:29.385 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-06-23T18:34:29.385 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-06-23T18:34:29.385 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-06-23T18:34:29.386 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-06-23T18:34:29.386 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-06-23T18:34:29.386 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-06-23T18:34:29.386 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-06-23T18:34:29.386 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-06-23T18:34:29.386 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-06-23T18:34:29.386 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-06-23T18:34:29.386 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-06-23T18:34:29.387 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-06-23T18:34:29.387 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-06-23T18:34:29.387 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-06-23T18:34:29.387 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-06-23T18:34:29.387 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-06-23T18:34:29.387 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-06-23T18:34:29.387 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-06-23T18:34:29.388 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-06-23T18:34:29.388 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-06-23T18:34:29.388 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-06-23T18:34:29.388 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-06-23T18:34:29.388 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-06-23T18:34:29.388 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-06-23T18:34:29.388 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-06-23T18:34:29.389 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-06-23T18:34:29.389 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-06-23T18:34:29.389 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 129/134 2023-06-23T18:34:29.389 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-06-23T18:34:29.389 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 131/134 2023-06-23T18:34:29.389 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:34:29.389 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-06-23T18:34:29.677 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 105/134 2023-06-23T18:34:30.636 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 105/134 2023-06-23T18:34:30.826 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 106/134 2023-06-23T18:34:30.917 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 106/134 2023-06-23T18:34:31.278 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el 107/134 2023-06-23T18:34:31.460 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 108/134 2023-06-23T18:34:31.494 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 108/134 2023-06-23T18:34:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 109/134 2023-06-23T18:34:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 109/134 2023-06-23T18:34:32.192 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:34:32.192 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:49 ago on Fri 23 Jun 2023 06:32:43 PM UTC. 2023-06-23T18:34:32.999 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 110/134 2023-06-23T18:34:33.222 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 111/134 2023-06-23T18:34:33.260 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 111/134 2023-06-23T18:34:33.418 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4479.g8627a 112/134 2023-06-23T18:34:33.459 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4479.g8627a 112/134 2023-06-23T18:34:33.634 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 113/134 2023-06-23T18:34:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 113/134 2023-06-23T18:34:34.134 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 114/134 2023-06-23T18:34:34.953 INFO:teuthology.orchestra.run.smithi155.stdout:Installed products updated. 2023-06-23T18:34:35.175 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:34:35.175 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2023-06-23T18:34:35.175 INFO:teuthology.orchestra.run.smithi155.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-06-23T18:34:35.175 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.175 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.176 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2023-06-23T18:34:35.176 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-06-23T18:34:35.176 INFO:teuthology.orchestra.run.smithi155.stdout: automake-1.16.1-7.el8.noarch 2023-06-23T18:34:35.176 INFO:teuthology.orchestra.run.smithi155.stdout: bison-3.0.4-10.el8.x86_64 2023-06-23T18:34:35.176 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.176 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.177 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.177 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.177 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:35.177 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.177 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.177 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.178 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:35.178 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:35.178 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:35.178 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:35.178 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:35.178 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.178 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.179 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:35.179 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.179 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.179 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.179 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:35.179 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:35.179 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.180 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:35.180 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-06-23T18:34:35.180 INFO:teuthology.orchestra.run.smithi155.stdout: flex-2.6.1-9.el8.x86_64 2023-06-23T18:34:35.180 INFO:teuthology.orchestra.run.smithi155.stdout: fuse-2.9.7-15.el8.x86_64 2023-06-23T18:34:35.180 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-23T18:34:35.180 INFO:teuthology.orchestra.run.smithi155.stdout: jq-1.6-3.el8.x86_64 2023-06-23T18:34:35.181 INFO:teuthology.orchestra.run.smithi155.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-06-23T18:34:35.181 INFO:teuthology.orchestra.run.smithi155.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-06-23T18:34:35.181 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-23T18:34:35.181 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.181 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.181 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.181 INFO:teuthology.orchestra.run.smithi155.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-06-23T18:34:35.182 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-06-23T18:34:35.182 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-23T18:34:35.182 INFO:teuthology.orchestra.run.smithi155.stdout: libpq-13.5-1.el8.x86_64 2023-06-23T18:34:35.182 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-06-23T18:34:35.182 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-23T18:34:35.182 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.182 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.183 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-23T18:34:35.183 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.183 INFO:teuthology.orchestra.run.smithi155.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-23T18:34:35.183 INFO:teuthology.orchestra.run.smithi155.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-06-23T18:34:35.183 INFO:teuthology.orchestra.run.smithi155.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-23T18:34:35.183 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-23T18:34:35.183 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-06-23T18:34:35.184 INFO:teuthology.orchestra.run.smithi155.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-23T18:34:35.184 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-23T18:34:35.184 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-23T18:34:35.184 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.15-3.el8.x86_64 2023-06-23T18:34:35.184 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-06-23T18:34:35.184 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-06-23T18:34:35.185 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-23T18:34:35.185 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-23T18:34:35.185 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-23T18:34:35.185 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:34:35.185 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:34:35.185 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:34:35.185 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-23T18:34:35.185 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-23T18:34:35.186 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-23T18:34:35.186 INFO:teuthology.orchestra.run.smithi155.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-23T18:34:35.186 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-23T18:34:35.186 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.186 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.186 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.186 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-23T18:34:35.187 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-06-23T18:34:35.187 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-23T18:34:35.187 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-23T18:34:35.187 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-06-23T18:34:35.187 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-23T18:34:35.187 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-23T18:34:35.188 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-23T18:34:35.188 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-23T18:34:35.188 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-23T18:34:35.188 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-23T18:34:35.188 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-23T18:34:35.188 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-23T18:34:35.188 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-23T18:34:35.189 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-23T18:34:35.189 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-23T18:34:35.189 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-23T18:34:35.189 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-23T18:34:35.189 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-23T18:34:35.189 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-23T18:34:35.189 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-23T18:34:35.190 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-23T18:34:35.190 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-2.6-1.el8.noarch 2023-06-23T18:34:35.190 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-23T18:34:35.190 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-23T18:34:35.190 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-23T18:34:35.190 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-23T18:34:35.190 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-23T18:34:35.191 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-23T18:34:35.191 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.191 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.191 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-23T18:34:35.191 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-23T18:34:35.191 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.191 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-23T18:34:35.192 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-23T18:34:35.192 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-06-23T18:34:35.192 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-23T18:34:35.192 INFO:teuthology.orchestra.run.smithi155.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-23T18:34:35.192 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-23T18:34:35.193 INFO:teuthology.orchestra.run.smithi155.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-23T18:34:35.193 INFO:teuthology.orchestra.run.smithi155.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-23T18:34:35.193 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-23T18:34:35.193 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-23T18:34:35.193 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-23T18:34:35.193 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-23T18:34:35.193 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-23T18:34:35.194 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.194 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.194 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:35.194 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-23T18:34:35.194 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-23T18:34:35.194 INFO:teuthology.orchestra.run.smithi155.stdout: uuid-1.6.2-43.el8.x86_64 2023-06-23T18:34:35.194 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-06-23T18:34:35.194 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-23T18:34:35.195 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-06-23T18:34:35.195 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:34:35.195 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2023-06-23T18:34:35.381 DEBUG:teuthology.parallel:result is None 2023-06-23T18:34:38.637 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 114/134 2023-06-23T18:34:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 115/134 2023-06-23T18:34:39.814 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-06-23T18:34:39.995 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 117/134 2023-06-23T18:34:40.293 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-06-23T18:34:40.449 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-06-23T18:34:40.639 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-06-23T18:34:40.878 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-06-23T18:34:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_ 121/134 2023-06-23T18:34:41.233 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-23T18:34:43.317 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:34:43.565 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:34:43.583 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:34:43.749 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 124/134 2023-06-23T18:34:44.098 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 125/134 2023-06-23T18:34:44.392 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 126/134 2023-06-23T18:34:44.784 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-06-23T18:34:44.981 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-06-23T18:34:45.086 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-06-23T18:34:45.281 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-06-23T18:34:45.294 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-06-23T18:34:45.448 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-23T18:34:45.456 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-23T18:34:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-06-23T18:34:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:34:45.801 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:34:45.801 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-23T18:34:46.062 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-23T18:34:46.239 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:34:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:34:46.543 INFO:teuthology.orchestra.run.smithi079.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T18:34:46.543 INFO:teuthology.orchestra.run.smithi079.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T18:34:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:34:46.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 1/134 2023-06-23T18:34:46.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 2/134 2023-06-23T18:34:46.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 3/134 2023-06-23T18:34:46.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 4/134 2023-06-23T18:34:46.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4479.g8627a 5/134 2023-06-23T18:34:46.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:34:46.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:34:46.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 8/134 2023-06-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 9/134 2023-06-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 10/134 2023-06-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 11/134 2023-06-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 12/134 2023-06-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 13/134 2023-06-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_ 14/134 2023-06-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 15/134 2023-06-23T18:34:46.771 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 16/134 2023-06-23T18:34:46.771 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_6 17/134 2023-06-23T18:34:46.771 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 18/134 2023-06-23T18:34:46.771 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 19/134 2023-06-23T18:34:46.772 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el 20/134 2023-06-23T18:34:46.772 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-common-2:18.0.0-4479.g8627af3e.el8. 21/134 2023-06-23T18:34:46.772 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_6 22/134 2023-06-23T18:34:46.773 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 23/134 2023-06-23T18:34:46.773 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 24/134 2023-06-23T18:34:46.773 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 25/134 2023-06-23T18:34:46.774 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 26/134 2023-06-23T18:34:46.774 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 27/134 2023-06-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 28/134 2023-06-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e. 29/134 2023-06-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 30/134 2023-06-23T18:34:46.785 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 31/134 2023-06-23T18:34:46.785 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 32/134 2023-06-23T18:34:46.785 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el 33/134 2023-06-23T18:34:46.786 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 34/134 2023-06-23T18:34:46.786 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.e 35/134 2023-06-23T18:34:46.786 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 36/134 2023-06-23T18:34:46.786 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 37/134 2023-06-23T18:34:46.787 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 38/134 2023-06-23T18:34:46.787 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-06-23T18:34:46.787 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-06-23T18:34:46.788 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-06-23T18:34:46.788 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-06-23T18:34:46.788 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-06-23T18:34:46.788 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-06-23T18:34:46.789 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-06-23T18:34:46.789 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-06-23T18:34:46.789 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-06-23T18:34:46.789 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-06-23T18:34:46.790 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-06-23T18:34:46.790 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-06-23T18:34:46.790 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-06-23T18:34:46.790 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-06-23T18:34:46.790 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-06-23T18:34:46.790 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-06-23T18:34:46.791 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-06-23T18:34:46.791 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-06-23T18:34:46.791 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-06-23T18:34:46.791 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-06-23T18:34:46.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-06-23T18:34:46.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-06-23T18:34:46.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-06-23T18:34:46.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-06-23T18:34:46.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-06-23T18:34:46.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-06-23T18:34:46.792 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-06-23T18:34:46.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-06-23T18:34:46.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-06-23T18:34:46.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-06-23T18:34:46.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-06-23T18:34:46.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-06-23T18:34:46.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-06-23T18:34:46.793 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-06-23T18:34:46.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-06-23T18:34:46.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-06-23T18:34:46.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-06-23T18:34:46.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-06-23T18:34:46.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-06-23T18:34:46.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-06-23T18:34:46.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-06-23T18:34:46.794 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-06-23T18:34:46.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-06-23T18:34:46.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-06-23T18:34:46.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-06-23T18:34:46.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-06-23T18:34:46.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-06-23T18:34:46.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-06-23T18:34:46.795 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-06-23T18:34:46.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-06-23T18:34:46.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-06-23T18:34:46.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-06-23T18:34:46.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-06-23T18:34:46.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-06-23T18:34:46.796 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-06-23T18:34:46.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-06-23T18:34:46.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-06-23T18:34:46.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-06-23T18:34:46.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-06-23T18:34:46.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-06-23T18:34:46.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-06-23T18:34:46.797 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-06-23T18:34:46.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-06-23T18:34:46.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-06-23T18:34:46.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-06-23T18:34:46.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-06-23T18:34:46.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-06-23T18:34:46.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-06-23T18:34:46.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-06-23T18:34:46.798 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-06-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-06-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-06-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-06-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-06-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-06-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-06-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-06-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-06-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-06-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-06-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-06-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-06-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-06-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-06-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-06-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-06-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-06-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-06-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-06-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-06-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 129/134 2023-06-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-06-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 131/134 2023-06-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-06-23T18:34:49.472 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:34:49.472 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:47 ago on Fri 23 Jun 2023 06:33:02 PM UTC. 2023-06-23T18:34:52.111 INFO:teuthology.orchestra.run.smithi079.stdout:Installed products updated. 2023-06-23T18:34:52.330 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:34:52.331 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2023-06-23T18:34:52.331 INFO:teuthology.orchestra.run.smithi079.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-06-23T18:34:52.331 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.331 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.332 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-06-23T18:34:52.332 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-06-23T18:34:52.332 INFO:teuthology.orchestra.run.smithi079.stdout: automake-1.16.1-7.el8.noarch 2023-06-23T18:34:52.332 INFO:teuthology.orchestra.run.smithi079.stdout: bison-3.0.4-10.el8.x86_64 2023-06-23T18:34:52.332 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.333 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.333 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.333 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.333 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:52.333 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.333 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.334 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.334 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:52.334 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:52.334 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:52.334 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:52.334 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:52.334 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.335 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.335 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:52.335 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.335 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.335 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.335 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:52.336 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:52.336 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.336 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:34:52.336 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-06-23T18:34:52.336 INFO:teuthology.orchestra.run.smithi079.stdout: flex-2.6.1-9.el8.x86_64 2023-06-23T18:34:52.336 INFO:teuthology.orchestra.run.smithi079.stdout: fuse-2.9.7-15.el8.x86_64 2023-06-23T18:34:52.337 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-23T18:34:52.337 INFO:teuthology.orchestra.run.smithi079.stdout: jq-1.6-3.el8.x86_64 2023-06-23T18:34:52.337 INFO:teuthology.orchestra.run.smithi079.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-06-23T18:34:52.337 INFO:teuthology.orchestra.run.smithi079.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-06-23T18:34:52.337 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-23T18:34:52.337 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.338 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.338 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.338 INFO:teuthology.orchestra.run.smithi079.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-06-23T18:34:52.338 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-06-23T18:34:52.338 INFO:teuthology.orchestra.run.smithi079.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-23T18:34:52.338 INFO:teuthology.orchestra.run.smithi079.stdout: libpq-13.5-1.el8.x86_64 2023-06-23T18:34:52.338 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-06-23T18:34:52.339 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-23T18:34:52.339 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.339 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.339 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-23T18:34:52.339 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.340 INFO:teuthology.orchestra.run.smithi079.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-23T18:34:52.340 INFO:teuthology.orchestra.run.smithi079.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-06-23T18:34:52.340 INFO:teuthology.orchestra.run.smithi079.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-23T18:34:52.340 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-23T18:34:52.340 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-06-23T18:34:52.341 INFO:teuthology.orchestra.run.smithi079.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-23T18:34:52.341 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-23T18:34:52.341 INFO:teuthology.orchestra.run.smithi079.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-23T18:34:52.341 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-0.3.15-3.el8.x86_64 2023-06-23T18:34:52.341 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-06-23T18:34:52.341 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-06-23T18:34:52.341 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-23T18:34:52.342 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-23T18:34:52.342 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-23T18:34:52.342 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:34:52.342 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:34:52.342 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:34:52.342 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-23T18:34:52.343 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-23T18:34:52.343 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-23T18:34:52.343 INFO:teuthology.orchestra.run.smithi079.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-23T18:34:52.343 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-23T18:34:52.343 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.344 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.344 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.344 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-23T18:34:52.344 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-06-23T18:34:52.344 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-23T18:34:52.345 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-23T18:34:52.345 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-06-23T18:34:52.345 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-23T18:34:52.345 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-23T18:34:52.345 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-23T18:34:52.345 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-23T18:34:52.345 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-23T18:34:52.346 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-23T18:34:52.346 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-23T18:34:52.346 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-23T18:34:52.346 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-23T18:34:52.346 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-23T18:34:52.346 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-23T18:34:52.346 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-23T18:34:52.347 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-23T18:34:52.347 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-23T18:34:52.347 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-23T18:34:52.347 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-23T18:34:52.347 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-23T18:34:52.347 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend-2.6-1.el8.noarch 2023-06-23T18:34:52.347 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-23T18:34:52.348 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-23T18:34:52.348 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-23T18:34:52.348 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-23T18:34:52.348 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-23T18:34:52.348 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-23T18:34:52.348 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.349 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.349 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-23T18:34:52.349 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-23T18:34:52.349 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.349 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-23T18:34:52.349 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-23T18:34:52.349 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-06-23T18:34:52.350 INFO:teuthology.orchestra.run.smithi079.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-23T18:34:52.350 INFO:teuthology.orchestra.run.smithi079.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-23T18:34:52.350 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-23T18:34:52.350 INFO:teuthology.orchestra.run.smithi079.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-23T18:34:52.350 INFO:teuthology.orchestra.run.smithi079.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-23T18:34:52.350 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-23T18:34:52.350 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-23T18:34:52.350 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-23T18:34:52.351 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-23T18:34:52.351 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-23T18:34:52.351 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.351 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.351 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:34:52.351 INFO:teuthology.orchestra.run.smithi079.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-23T18:34:52.351 INFO:teuthology.orchestra.run.smithi079.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-23T18:34:52.352 INFO:teuthology.orchestra.run.smithi079.stdout: uuid-1.6.2-43.el8.x86_64 2023-06-23T18:34:52.352 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-06-23T18:34:52.352 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-23T18:34:52.352 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-06-23T18:34:52.353 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:34:52.353 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-06-23T18:34:52.520 DEBUG:teuthology.parallel:result is None 2023-06-23T18:34:52.520 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:34:53.865 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-23T18:34:53.893 INFO:teuthology.orchestra.run.smithi079.stdout:18.0.0-4479.g8627af3e.el8 2023-06-23T18:34:53.894 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4479.g8627af3e.el8 2023-06-23T18:34:53.894 INFO:teuthology.task.install:The correct ceph version 18.0.0-4479.g8627af3e is installed. 2023-06-23T18:34:53.895 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:34:57.126 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-23T18:34:57.155 INFO:teuthology.orchestra.run.smithi155.stdout:18.0.0-4479.g8627af3e.el8 2023-06-23T18:34:57.156 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4479.g8627af3e.el8 2023-06-23T18:34:57.157 INFO:teuthology.task.install:The correct ceph version 18.0.0-4479.g8627af3e is installed. 2023-06-23T18:34:57.157 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-06-23T18:34:57.158 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:34:57.158 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-23T18:34:57.205 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:34:57.206 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-23T18:34:57.252 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-06-23T18:34:57.252 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:34:57.252 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2023-06-23T18:34:57.282 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-23T18:34:57.351 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:34:57.351 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2023-06-23T18:34:57.381 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-23T18:34:57.449 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-06-23T18:34:57.449 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:34:57.450 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-23T18:34:57.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-23T18:34:57.545 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:34:57.545 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-23T18:34:57.574 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-23T18:34:57.642 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-06-23T18:34:57.643 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:34:57.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/stdin-killer 2023-06-23T18:34:57.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-23T18:34:57.739 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:34:57.739 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2023-06-23T18:34:57.767 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-23T18:34:57.834 INFO:teuthology.run_tasks:Running task ceph... 2023-06-23T18:34:57.918 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-06-23T18:34:57.919 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 777 /var/log/ceph 2023-06-23T18:34:57.921 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /var/log/ceph 2023-06-23T18:34:57.947 INFO:tasks.ceph:Disabling ceph logrotate... 2023-06-23T18:34:57.948 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-23T18:34:57.989 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-23T18:34:58.015 INFO:tasks.ceph:Creating extra log directories... 2023-06-23T18:34:58.015 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-23T18:34:58.056 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-23T18:34:58.089 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-06-23T18:34:58.089 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-06-23T18:34:58.089 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-06-23T18:34:58.090 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:34:58.090 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-23T18:34:58.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-23T18:34:58.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-23T18:34:58.504 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:34:58.504 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-23T18:34:58.522 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-23T18:34:58.664 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-23T18:34:58.697 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-06-23T18:34:58.697 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-06-23T18:34:58.698 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886', 'branch': 'wip-rishabh-improvements-authmon', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota}', 'email': None, 'first_in_suite': False, 'job_id': '7313886', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-improvements-authmon'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-improvements-authmon', 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-improvements-authmon', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi079.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi155.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_quota']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '076bbebc42a14f7d568aaa78eabb0038327bcb23', 'timestamp': '2023-06-23_17:37:30', 'tube': 'smithi', 'user': 'rishabh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-06-23T18:34:58.699 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-23T18:34:58.702 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-23T18:34:58.748 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/run/ceph 2023-06-23T18:34:58.761 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m0777 -- /var/run/ceph 2023-06-23T18:34:58.816 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:34:58.816 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2023-06-23T18:34:58.849 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-23T18:34:58.850 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 2023-06-23T18:34:58.907 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-23T18:34:58.907 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:34:58.908 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 378955 Links: 1 2023-06-23T18:34:58.908 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:34:58.908 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:34:58.908 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2023-06-23 18:34:46.448110453 +0000 2023-06-23T18:34:58.908 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2023-06-23 18:31:55.303488575 +0000 2023-06-23T18:34:58.908 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2023-06-23 18:31:55.303488575 +0000 2023-06-23T18:34:58.908 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2023-06-23T18:34:58.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-23T18:34:58.973 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2023-06-23T18:34:58.973 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2023-06-23T18:34:58.974 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000242957 s, 2.1 MB/s 2023-06-23T18:34:58.975 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-23T18:34:59.032 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 2023-06-23T18:34:59.089 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-23T18:34:59.089 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:34:59.090 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 378060 Links: 1 2023-06-23T18:34:59.090 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:34:59.090 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:34:59.090 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2023-06-23 18:34:46.448110453 +0000 2023-06-23T18:34:59.090 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2023-06-23 18:31:55.825476136 +0000 2023-06-23T18:34:59.090 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2023-06-23 18:31:55.825476136 +0000 2023-06-23T18:34:59.090 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2023-06-23T18:34:59.091 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-23T18:34:59.154 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2023-06-23T18:34:59.154 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2023-06-23T18:34:59.155 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000219861 s, 2.3 MB/s 2023-06-23T18:34:59.156 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-23T18:34:59.213 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 2023-06-23T18:34:59.270 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-23T18:34:59.270 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:34:59.270 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 377285 Links: 1 2023-06-23T18:34:59.270 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:34:59.270 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:34:59.270 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2023-06-23 18:34:46.448110453 +0000 2023-06-23T18:34:59.271 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2023-06-23 18:31:56.337463935 +0000 2023-06-23T18:34:59.271 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2023-06-23 18:31:56.337463935 +0000 2023-06-23T18:34:59.271 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2023-06-23T18:34:59.271 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-23T18:34:59.334 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2023-06-23T18:34:59.334 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2023-06-23T18:34:59.334 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000234665 s, 2.2 MB/s 2023-06-23T18:34:59.335 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-23T18:34:59.392 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 2023-06-23T18:34:59.449 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-23T18:34:59.449 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:34:59.449 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 371473 Links: 1 2023-06-23T18:34:59.450 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:34:59.450 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:34:59.450 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2023-06-23 18:34:46.448110453 +0000 2023-06-23T18:34:59.450 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2023-06-23 18:31:56.865451354 +0000 2023-06-23T18:34:59.450 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2023-06-23 18:31:56.865451354 +0000 2023-06-23T18:34:59.450 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2023-06-23T18:34:59.451 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-23T18:34:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2023-06-23T18:34:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2023-06-23T18:34:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000236594 s, 2.2 MB/s 2023-06-23T18:34:59.518 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-23T18:34:59.575 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T18:34:59.579 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:34:59.579 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2023-06-23T18:34:59.600 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-23T18:34:59.601 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2023-06-23T18:34:59.661 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-23T18:34:59.661 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:34:59.661 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 371463 Links: 1 2023-06-23T18:34:59.661 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:34:59.662 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:34:59.662 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-06-23 18:34:29.057508959 +0000 2023-06-23T18:34:59.662 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-06-23 18:31:48.559221804 +0000 2023-06-23T18:34:59.662 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-06-23 18:31:48.559221804 +0000 2023-06-23T18:34:59.662 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2023-06-23T18:34:59.663 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-23T18:34:59.731 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-06-23T18:34:59.731 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-06-23T18:34:59.731 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000261143 s, 2.0 MB/s 2023-06-23T18:34:59.732 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-23T18:34:59.793 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2023-06-23T18:34:59.852 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-23T18:34:59.853 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:34:59.853 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 377025 Links: 1 2023-06-23T18:34:59.853 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:34:59.853 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:34:59.854 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-06-23 18:34:29.057508959 +0000 2023-06-23T18:34:59.854 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-06-23 18:31:49.087210844 +0000 2023-06-23T18:34:59.854 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-06-23 18:31:49.087210844 +0000 2023-06-23T18:34:59.854 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2023-06-23T18:34:59.854 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-23T18:34:59.920 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-06-23T18:34:59.921 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-06-23T18:34:59.921 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000244994 s, 2.1 MB/s 2023-06-23T18:34:59.922 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-23T18:34:59.982 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2023-06-23T18:35:00.040 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-23T18:35:00.041 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:35:00.041 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 378339 Links: 1 2023-06-23T18:35:00.041 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:35:00.041 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:35:00.041 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-06-23 18:34:29.057508959 +0000 2023-06-23T18:35:00.041 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-06-23 18:31:49.652199116 +0000 2023-06-23T18:35:00.042 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-06-23 18:31:49.652199116 +0000 2023-06-23T18:35:00.042 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2023-06-23T18:35:00.042 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-23T18:35:00.108 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-06-23T18:35:00.109 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-06-23T18:35:00.109 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000315753 s, 1.6 MB/s 2023-06-23T18:35:00.110 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-23T18:35:00.169 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2023-06-23T18:35:00.226 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-23T18:35:00.226 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:35:00.227 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 378382 Links: 1 2023-06-23T18:35:00.227 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:35:00.227 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:35:00.227 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-06-23 18:34:29.057508959 +0000 2023-06-23T18:35:00.227 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-06-23 18:31:50.167188426 +0000 2023-06-23T18:35:00.227 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-06-23 18:31:50.167188426 +0000 2023-06-23T18:35:00.227 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2023-06-23T18:35:00.228 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-23T18:35:00.293 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-06-23T18:35:00.293 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-06-23T18:35:00.293 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000233543 s, 2.2 MB/s 2023-06-23T18:35:00.294 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-23T18:35:00.352 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T18:35:00.353 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi155.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-23T18:35:00.353 INFO:tasks.ceph:Generating config... 2023-06-23T18:35:00.354 INFO:tasks.ceph:[client] client mount timeout = 600 2023-06-23T18:35:00.354 INFO:tasks.ceph:[client] debug client = 20 2023-06-23T18:35:00.355 INFO:tasks.ceph:[client] debug ms = 1 2023-06-23T18:35:00.355 INFO:tasks.ceph:[client] pid file = 2023-06-23T18:35:00.355 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-06-23T18:35:00.355 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-06-23T18:35:00.355 INFO:tasks.ceph:[mds] debug mds = 20 2023-06-23T18:35:00.356 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-06-23T18:35:00.356 INFO:tasks.ceph:[mds] debug ms = 1 2023-06-23T18:35:00.356 INFO:tasks.ceph:[mds] mds debug frag = True 2023-06-23T18:35:00.356 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-06-23T18:35:00.356 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-06-23T18:35:00.357 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-06-23T18:35:00.357 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-06-23T18:35:00.357 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-06-23T18:35:00.357 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-06-23T18:35:00.357 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-06-23T18:35:00.358 INFO:tasks.ceph:[mgr] debug ms = 1 2023-06-23T18:35:00.358 INFO:tasks.ceph:[mon] debug mon = 20 2023-06-23T18:35:00.358 INFO:tasks.ceph:[mon] debug ms = 1 2023-06-23T18:35:00.358 INFO:tasks.ceph:[mon] debug paxos = 20 2023-06-23T18:35:00.359 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-06-23T18:35:00.359 INFO:tasks.ceph:[osd] bdev async discard = True 2023-06-23T18:35:00.359 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-06-23T18:35:00.359 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-06-23T18:35:00.359 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-06-23T18:35:00.359 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-06-23T18:35:00.360 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-06-23T18:35:00.360 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-06-23T18:35:00.360 INFO:tasks.ceph:[osd] debug ms = 1 2023-06-23T18:35:00.360 INFO:tasks.ceph:[osd] debug osd = 20 2023-06-23T18:35:00.360 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-06-23T18:35:00.361 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-06-23T18:35:00.361 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-06-23T18:35:00.361 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-06-23T18:35:00.361 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-06-23T18:35:00.361 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-06-23T18:35:00.362 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-06-23T18:35:00.362 INFO:tasks.ceph:Setting up mon.a... 2023-06-23T18:35:00.362 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-06-23T18:35:00.415 INFO:teuthology.orchestra.run.smithi079.stdout:creating /etc/ceph/ceph.keyring 2023-06-23T18:35:00.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-06-23T18:35:00.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-23T18:35:00.496 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.79'), ('mon.b', '172.21.15.155'), ('mon.c', '[v2:172.21.15.155:3301,v1:172.21.15.155:6790]')] 2023-06-23T18:35:00.498 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.79,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-06-23T18:35:00.500 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:00.501 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-06-23T18:35:00.554 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.79 --add b 172.21.15.155 --addv c '[v2:172.21.15.155:3301,v1:172.21.15.155:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:35:00.641 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:35:00.641 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: generated fsid c6d43825-8b0e-40e8-8d7f-a5ab36592233 2023-06-23T18:35:00.642 INFO:teuthology.orchestra.run.smithi079.stdout:setting min_mon_release = pacific 2023-06-23T18:35:00.642 INFO:teuthology.orchestra.run.smithi079.stdout:epoch 0 2023-06-23T18:35:00.642 INFO:teuthology.orchestra.run.smithi079.stdout:fsid c6d43825-8b0e-40e8-8d7f-a5ab36592233 2023-06-23T18:35:00.642 INFO:teuthology.orchestra.run.smithi079.stdout:last_changed 2023-06-23T18:35:00.637781+0000 2023-06-23T18:35:00.642 INFO:teuthology.orchestra.run.smithi079.stdout:created 2023-06-23T18:35:00.637781+0000 2023-06-23T18:35:00.642 INFO:teuthology.orchestra.run.smithi079.stdout:min_mon_release 16 (pacific) 2023-06-23T18:35:00.642 INFO:teuthology.orchestra.run.smithi079.stdout:election_strategy: 1 2023-06-23T18:35:00.643 INFO:teuthology.orchestra.run.smithi079.stdout:0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2023-06-23T18:35:00.643 INFO:teuthology.orchestra.run.smithi079.stdout:1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.b 2023-06-23T18:35:00.643 INFO:teuthology.orchestra.run.smithi079.stdout:2: [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] mon.c 2023-06-23T18:35:00.643 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-06-23T18:35:00.644 DEBUG:teuthology.orchestra.run.smithi079:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-06-23T18:35:00.660 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c6d43825-8b0e-40e8-8d7f-a5ab36592233... 2023-06-23T18:35:00.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-23T18:35:00.703 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-23T18:35:00.745 INFO:teuthology.orchestra.run.smithi079.stdout:[global] 2023-06-23T18:35:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: chdir = "" 2023-06-23T18:35:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-23T18:35:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: auth supported = cephx 2023-06-23T18:35:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: filestore xattr use omap = true 2023-06-23T18:35:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: mon clock drift allowed = 1.000 2023-06-23T18:35:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: osd crush chooseleaf type = 0 2023-06-23T18:35:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: auth debug = true 2023-06-23T18:35:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on old message = true 2023-06-23T18:35:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on bug = true 2023-06-23T18:35:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-23T18:35:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: mon pg warn max object skew = 0 2023-06-23T18:35:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: # disable pg_autoscaler by default for new pools 2023-06-23T18:35:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-23T18:35:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default size = 2 2023-06-23T18:35:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow primary affinity = true 2023-06-23T18:35:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow pg remap = true 2023-06-23T18:35:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on legacy crush tunables = false 2023-06-23T18:35:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on crush straw calc version zero = false 2023-06-23T18:35:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on no sortbitwise = false 2023-06-23T18:35:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on osd down out interval zero = false 2023-06-23T18:35:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on too few osds = false 2023-06-23T18:35:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-23T18:35:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-23T18:35:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: mon_allow_pool_size_one = true 2023-06-23T18:35:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-23T18:35:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: osd default data pool replay window = 5 2023-06-23T18:35:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: mon allow pool delete = true 2023-06-23T18:35:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: mon cluster log file level = debug 2023-06-23T18:35:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: debug asserts on shutdown = true 2023-06-23T18:35:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: mon health detail to clog = false 2023-06-23T18:35:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: mon host = "172.21.15.79,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2023-06-23T18:35:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: fsid = c6d43825-8b0e-40e8-8d7f-a5ab36592233 2023-06-23T18:35:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.752 INFO:teuthology.orchestra.run.smithi079.stdout:[osd] 2023-06-23T18:35:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: osd journal size = 100 2023-06-23T18:35:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub load threshold = 5.0 2023-06-23T18:35:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub max interval = 600 2023-06-23T18:35:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: osd mclock profile = high_recovery_ops 2023-06-23T18:35:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: osd recover clone overlap = true 2023-06-23T18:35:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: osd recovery max chunk = 1048576 2023-06-23T18:35:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug shutdown = true 2023-06-23T18:35:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug op order = true 2023-06-23T18:35:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug verify stray on activate = true 2023-06-23T18:35:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: osd open classes on start = true 2023-06-23T18:35:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug pg log writeout = true 2023-06-23T18:35:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: osd deep scrub update digest min age = 30 2023-06-23T18:35:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: osd map max advance = 10 2023-06-23T18:35:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: journal zero on create = true 2023-06-23T18:35:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: filestore ondisk finisher threads = 3 2023-06-23T18:35:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: filestore apply finisher threads = 3 2023-06-23T18:35:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: bdev debug aio = true 2023-06-23T18:35:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug misdirected ops = true 2023-06-23T18:35:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: bdev async discard = True 2023-06-23T18:35:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: bdev enable discard = True 2023-06-23T18:35:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore allocator = bitmap 2023-06-23T18:35:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore block size = 96636764160 2023-06-23T18:35:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore fsck on mount = True 2023-06-23T18:35:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluefs = 1/20 2023-06-23T18:35:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluestore = 1/20 2023-06-23T18:35:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2023-06-23T18:35:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: debug osd = 20 2023-06-23T18:35:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: debug rocksdb = 4/10 2023-06-23T18:35:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd backfillfull_ratio = 0.85 2023-06-23T18:35:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd full ratio = 0.9 2023-06-23T18:35:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd nearfull ratio = 0.8 2023-06-23T18:35:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: osd failsafe full ratio = 0.95 2023-06-23T18:35:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: osd objectstore = bluestore 2023-06-23T18:35:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: osd op complaint time = 180 2023-06-23T18:35:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.759 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr] 2023-06-23T18:35:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2023-06-23T18:35:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: debug mgr = 20 2023-06-23T18:35:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2023-06-23T18:35:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2023-06-23T18:35:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2023-06-23T18:35:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2023-06-23T18:35:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: mgr/telemetry/nag = false 2023-06-23T18:35:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.760 INFO:teuthology.orchestra.run.smithi079.stdout:[mon] 2023-06-23T18:35:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2023-06-23T18:35:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2023-06-23T18:35:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: debug paxos = 20 2023-06-23T18:35:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2023-06-23T18:35:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: mon data avail warn = 5 2023-06-23T18:35:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: mon mgr mkfs grace = 240 2023-06-23T18:35:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2023-06-23T18:35:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd reporter subtree level = osd 2023-06-23T18:35:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd prime pg temp = true 2023-06-23T18:35:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2023-06-23T18:35:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-23T18:35:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: auth mon ticket ttl = 660 # 11m 2023-06-23T18:35:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: auth service ticket ttl = 240 # 4m 2023-06-23T18:35:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: # don't complain about insecure global_id in the test suite 2023-06-23T18:35:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-23T18:35:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-23T18:35:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: # 1m isn't quite enough 2023-06-23T18:35:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: mon_down_mkfs_grace = 2m 2023-06-23T18:35:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_filestore_osds = false 2023-06-23T18:35:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: mon op complaint time = 120 2023-06-23T18:35:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:00.764 INFO:teuthology.orchestra.run.smithi079.stdout:[client] 2023-06-23T18:35:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: rgw cache enabled = true 2023-06-23T18:35:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable ops log = true 2023-06-23T18:35:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable usage log = true 2023-06-23T18:35:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-23T18:35:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-23T18:35:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: client mount timeout = 600 2023-06-23T18:35:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: debug client = 20 2023-06-23T18:35:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2023-06-23T18:35:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: pid file = "" 2023-06-23T18:35:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: rados mon op timeout = 900 2023-06-23T18:35:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: rados osd op timeout = 900 2023-06-23T18:35:00.765 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.a] 2023-06-23T18:35:00.765 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.b] 2023-06-23T18:35:00.766 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.c] 2023-06-23T18:35:00.766 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.a] 2023-06-23T18:35:00.766 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.c] 2023-06-23T18:35:00.766 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.b] 2023-06-23T18:35:00.766 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.d] 2023-06-23T18:35:00.766 INFO:teuthology.orchestra.run.smithi079.stdout:[mds] 2023-06-23T18:35:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: debug mds = 20 2023-06-23T18:35:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: debug mds balancer = 20 2023-06-23T18:35:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2023-06-23T18:35:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: mds debug frag = True 2023-06-23T18:35:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: mds debug scatterstat = True 2023-06-23T18:35:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: mds op complaint time = 180 2023-06-23T18:35:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: mds verify scatter = True 2023-06-23T18:35:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: osd op complaint time = 180 2023-06-23T18:35:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: rados mon op timeout = 900 2023-06-23T18:35:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: rados osd op timeout = 900 2023-06-23T18:35:00.769 INFO:teuthology.orchestra.run.smithi155.stdout:[global] 2023-06-23T18:35:00.769 INFO:teuthology.orchestra.run.smithi155.stdout: chdir = "" 2023-06-23T18:35:00.769 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-23T18:35:00.770 INFO:teuthology.orchestra.run.smithi155.stdout: auth supported = cephx 2023-06-23T18:35:00.770 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.770 INFO:teuthology.orchestra.run.smithi155.stdout: filestore xattr use omap = true 2023-06-23T18:35:00.770 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.770 INFO:teuthology.orchestra.run.smithi155.stdout: mon clock drift allowed = 1.000 2023-06-23T18:35:00.770 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.771 INFO:teuthology.orchestra.run.smithi155.stdout: osd crush chooseleaf type = 0 2023-06-23T18:35:00.771 INFO:teuthology.orchestra.run.smithi155.stdout: auth debug = true 2023-06-23T18:35:00.771 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.771 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on old message = true 2023-06-23T18:35:00.771 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on bug = true 2023-06-23T18:35:00.771 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.771 INFO:teuthology.orchestra.run.smithi155.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-23T18:35:00.772 INFO:teuthology.orchestra.run.smithi155.stdout: mon pg warn max object skew = 0 2023-06-23T18:35:00.772 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.772 INFO:teuthology.orchestra.run.smithi155.stdout: # disable pg_autoscaler by default for new pools 2023-06-23T18:35:00.772 INFO:teuthology.orchestra.run.smithi155.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-23T18:35:00.772 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.772 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default size = 2 2023-06-23T18:35:00.772 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.773 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow primary affinity = true 2023-06-23T18:35:00.773 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow pg remap = true 2023-06-23T18:35:00.773 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on legacy crush tunables = false 2023-06-23T18:35:00.773 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on crush straw calc version zero = false 2023-06-23T18:35:00.773 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on no sortbitwise = false 2023-06-23T18:35:00.773 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on osd down out interval zero = false 2023-06-23T18:35:00.773 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on too few osds = false 2023-06-23T18:35:00.774 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-23T18:35:00.774 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-23T18:35:00.774 INFO:teuthology.orchestra.run.smithi155.stdout: mon_allow_pool_size_one = true 2023-06-23T18:35:00.774 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.774 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-23T18:35:00.774 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.774 INFO:teuthology.orchestra.run.smithi155.stdout: osd default data pool replay window = 5 2023-06-23T18:35:00.775 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.775 INFO:teuthology.orchestra.run.smithi155.stdout: mon allow pool delete = true 2023-06-23T18:35:00.775 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.775 INFO:teuthology.orchestra.run.smithi155.stdout: mon cluster log file level = debug 2023-06-23T18:35:00.775 INFO:teuthology.orchestra.run.smithi155.stdout: debug asserts on shutdown = true 2023-06-23T18:35:00.775 INFO:teuthology.orchestra.run.smithi155.stdout: mon health detail to clog = false 2023-06-23T18:35:00.775 INFO:teuthology.orchestra.run.smithi155.stdout: mon host = "172.21.15.79,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2023-06-23T18:35:00.776 INFO:teuthology.orchestra.run.smithi155.stdout: fsid = c6d43825-8b0e-40e8-8d7f-a5ab36592233 2023-06-23T18:35:00.776 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.776 INFO:teuthology.orchestra.run.smithi155.stdout:[osd] 2023-06-23T18:35:00.776 INFO:teuthology.orchestra.run.smithi155.stdout: osd journal size = 100 2023-06-23T18:35:00.776 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.776 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub load threshold = 5.0 2023-06-23T18:35:00.776 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub max interval = 600 2023-06-23T18:35:00.777 INFO:teuthology.orchestra.run.smithi155.stdout: osd mclock profile = high_recovery_ops 2023-06-23T18:35:00.777 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.777 INFO:teuthology.orchestra.run.smithi155.stdout: osd recover clone overlap = true 2023-06-23T18:35:00.777 INFO:teuthology.orchestra.run.smithi155.stdout: osd recovery max chunk = 1048576 2023-06-23T18:35:00.777 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.777 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug shutdown = true 2023-06-23T18:35:00.778 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug op order = true 2023-06-23T18:35:00.778 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug verify stray on activate = true 2023-06-23T18:35:00.778 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.778 INFO:teuthology.orchestra.run.smithi155.stdout: osd open classes on start = true 2023-06-23T18:35:00.778 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug pg log writeout = true 2023-06-23T18:35:00.778 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.778 INFO:teuthology.orchestra.run.smithi155.stdout: osd deep scrub update digest min age = 30 2023-06-23T18:35:00.779 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.779 INFO:teuthology.orchestra.run.smithi155.stdout: osd map max advance = 10 2023-06-23T18:35:00.779 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.779 INFO:teuthology.orchestra.run.smithi155.stdout: journal zero on create = true 2023-06-23T18:35:00.779 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.779 INFO:teuthology.orchestra.run.smithi155.stdout: filestore ondisk finisher threads = 3 2023-06-23T18:35:00.779 INFO:teuthology.orchestra.run.smithi155.stdout: filestore apply finisher threads = 3 2023-06-23T18:35:00.780 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.780 INFO:teuthology.orchestra.run.smithi155.stdout: bdev debug aio = true 2023-06-23T18:35:00.780 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug misdirected ops = true 2023-06-23T18:35:00.780 INFO:teuthology.orchestra.run.smithi155.stdout: bdev async discard = True 2023-06-23T18:35:00.780 INFO:teuthology.orchestra.run.smithi155.stdout: bdev enable discard = True 2023-06-23T18:35:00.780 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore allocator = bitmap 2023-06-23T18:35:00.780 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore block size = 96636764160 2023-06-23T18:35:00.781 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore fsck on mount = True 2023-06-23T18:35:00.781 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluefs = 1/20 2023-06-23T18:35:00.781 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluestore = 1/20 2023-06-23T18:35:00.781 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2023-06-23T18:35:00.781 INFO:teuthology.orchestra.run.smithi155.stdout: debug osd = 20 2023-06-23T18:35:00.781 INFO:teuthology.orchestra.run.smithi155.stdout: debug rocksdb = 4/10 2023-06-23T18:35:00.781 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd backfillfull_ratio = 0.85 2023-06-23T18:35:00.782 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd full ratio = 0.9 2023-06-23T18:35:00.782 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd nearfull ratio = 0.8 2023-06-23T18:35:00.782 INFO:teuthology.orchestra.run.smithi155.stdout: osd failsafe full ratio = 0.95 2023-06-23T18:35:00.782 INFO:teuthology.orchestra.run.smithi155.stdout: osd objectstore = bluestore 2023-06-23T18:35:00.782 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2023-06-23T18:35:00.782 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.782 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr] 2023-06-23T18:35:00.783 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2023-06-23T18:35:00.783 INFO:teuthology.orchestra.run.smithi155.stdout: debug mgr = 20 2023-06-23T18:35:00.783 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2023-06-23T18:35:00.783 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2023-06-23T18:35:00.783 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2023-06-23T18:35:00.783 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2023-06-23T18:35:00.783 INFO:teuthology.orchestra.run.smithi155.stdout: mgr/telemetry/nag = false 2023-06-23T18:35:00.784 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.784 INFO:teuthology.orchestra.run.smithi155.stdout:[mon] 2023-06-23T18:35:00.784 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2023-06-23T18:35:00.784 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2023-06-23T18:35:00.784 INFO:teuthology.orchestra.run.smithi155.stdout: debug paxos = 20 2023-06-23T18:35:00.784 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2023-06-23T18:35:00.784 INFO:teuthology.orchestra.run.smithi155.stdout: mon data avail warn = 5 2023-06-23T18:35:00.785 INFO:teuthology.orchestra.run.smithi155.stdout: mon mgr mkfs grace = 240 2023-06-23T18:35:00.785 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2023-06-23T18:35:00.785 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd reporter subtree level = osd 2023-06-23T18:35:00.785 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd prime pg temp = true 2023-06-23T18:35:00.785 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2023-06-23T18:35:00.785 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.785 INFO:teuthology.orchestra.run.smithi155.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-23T18:35:00.786 INFO:teuthology.orchestra.run.smithi155.stdout: auth mon ticket ttl = 660 # 11m 2023-06-23T18:35:00.786 INFO:teuthology.orchestra.run.smithi155.stdout: auth service ticket ttl = 240 # 4m 2023-06-23T18:35:00.786 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.786 INFO:teuthology.orchestra.run.smithi155.stdout: # don't complain about insecure global_id in the test suite 2023-06-23T18:35:00.786 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-23T18:35:00.786 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-23T18:35:00.786 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.786 INFO:teuthology.orchestra.run.smithi155.stdout: # 1m isn't quite enough 2023-06-23T18:35:00.787 INFO:teuthology.orchestra.run.smithi155.stdout: mon_down_mkfs_grace = 2m 2023-06-23T18:35:00.787 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.787 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_filestore_osds = false 2023-06-23T18:35:00.787 INFO:teuthology.orchestra.run.smithi155.stdout: mon op complaint time = 120 2023-06-23T18:35:00.787 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-06-23T18:35:00.787 INFO:teuthology.orchestra.run.smithi155.stdout:[client] 2023-06-23T18:35:00.787 INFO:teuthology.orchestra.run.smithi155.stdout: rgw cache enabled = true 2023-06-23T18:35:00.788 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable ops log = true 2023-06-23T18:35:00.788 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable usage log = true 2023-06-23T18:35:00.788 INFO:teuthology.orchestra.run.smithi155.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-23T18:35:00.788 INFO:teuthology.orchestra.run.smithi155.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-23T18:35:00.788 INFO:teuthology.orchestra.run.smithi155.stdout: client mount timeout = 600 2023-06-23T18:35:00.788 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2023-06-23T18:35:00.789 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2023-06-23T18:35:00.789 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = "" 2023-06-23T18:35:00.789 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2023-06-23T18:35:00.789 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2023-06-23T18:35:00.789 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.a] 2023-06-23T18:35:00.790 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.b] 2023-06-23T18:35:00.790 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.c] 2023-06-23T18:35:00.790 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.a] 2023-06-23T18:35:00.790 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.c] 2023-06-23T18:35:00.790 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.b] 2023-06-23T18:35:00.790 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.d] 2023-06-23T18:35:00.790 INFO:teuthology.orchestra.run.smithi155.stdout:[mds] 2023-06-23T18:35:00.791 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds = 20 2023-06-23T18:35:00.791 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds balancer = 20 2023-06-23T18:35:00.791 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2023-06-23T18:35:00.791 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug frag = True 2023-06-23T18:35:00.791 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug scatterstat = True 2023-06-23T18:35:00.791 INFO:teuthology.orchestra.run.smithi155.stdout: mds op complaint time = 180 2023-06-23T18:35:00.791 INFO:teuthology.orchestra.run.smithi155.stdout: mds verify scatter = True 2023-06-23T18:35:00.791 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2023-06-23T18:35:00.792 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2023-06-23T18:35:00.792 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2023-06-23T18:35:00.792 INFO:tasks.ceph:Creating admin key on mon.a... 2023-06-23T18:35:00.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-06-23T18:35:00.867 INFO:tasks.ceph:Copying monmap to all nodes... 2023-06-23T18:35:00.867 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:00.868 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-06-23T18:35:00.884 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:00.885 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-06-23T18:35:00.940 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:35:00.941 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:00.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-23T18:35:00.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-23T18:35:01.016 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:01.018 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:35:01.074 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:35:01.074 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:01.074 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-23T18:35:01.075 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-23T18:35:01.117 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:01.118 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:35:01.175 INFO:tasks.ceph:Setting up mon nodes... 2023-06-23T18:35:01.176 INFO:tasks.ceph:Setting up mgr nodes... 2023-06-23T18:35:01.176 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-06-23T18:35:01.229 INFO:teuthology.orchestra.run.smithi079.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-06-23T18:35:01.232 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-06-23T18:35:01.304 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-06-23T18:35:01.307 INFO:tasks.ceph:Setting up mds nodes... 2023-06-23T18:35:01.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-06-23T18:35:01.360 INFO:teuthology.orchestra.run.smithi079.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-06-23T18:35:01.363 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-06-23T18:35:01.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-06-23T18:35:01.482 INFO:teuthology.orchestra.run.smithi079.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-06-23T18:35:01.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-06-23T18:35:01.510 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-06-23T18:35:01.566 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-06-23T18:35:01.569 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-06-23T18:35:01.595 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-06-23T18:35:01.689 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-06-23T18:35:01.692 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-06-23T18:35:01.718 INFO:tasks.ceph_client:Setting up client nodes... 2023-06-23T18:35:01.719 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-06-23T18:35:01.762 INFO:teuthology.orchestra.run.smithi079.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-06-23T18:35:01.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-06-23T18:35:01.817 INFO:teuthology.orchestra.run.smithi079.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-06-23T18:35:01.829 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-06-23T18:35:01.872 INFO:teuthology.orchestra.run.smithi155.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-06-23T18:35:01.885 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-06-23T18:35:01.968 INFO:teuthology.orchestra.run.smithi155.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-06-23T18:35:01.981 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-06-23T18:35:01.982 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi155.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-23T18:35:01.982 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-06-23T18:35:02.008 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T18:35:02.009 INFO:tasks.ceph:role: osd.0 2023-06-23T18:35:02.009 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:35:02.009 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-23T18:35:02.082 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:35:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:35:02.083 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:35:02.083 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-23T18:35:02.083 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:35:02.083 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:35:02.083 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:35:02.083 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:35:02.083 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:35:02.084 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:35:02.507 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2023-06-23T18:35:02.508 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2023-06-23T18:35:02.509 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-06-23T18:35:02.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-06-23T18:35:02.717 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-06-23T18:35:02.784 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T18:35:02.784 INFO:tasks.ceph:role: osd.1 2023-06-23T18:35:02.784 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:35:02.785 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-23T18:35:02.858 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:35:02.859 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:35:02.859 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:35:02.859 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-23T18:35:02.859 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:35:02.859 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:35:02.859 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:35:02.860 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:35:02.860 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:35:02.860 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:35:03.284 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2023-06-23T18:35:03.286 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2023-06-23T18:35:03.286 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-06-23T18:35:03.391 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-06-23T18:35:03.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-06-23T18:35:03.488 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T18:35:03.488 INFO:tasks.ceph:role: osd.2 2023-06-23T18:35:03.488 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:35:03.489 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-23T18:35:03.561 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:35:03.561 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:35:03.561 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:35:03.562 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-23T18:35:03.562 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:35:03.562 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:35:03.562 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:35:03.562 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:35:03.562 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:35:03.562 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:35:04.002 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2023-06-23T18:35:04.003 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2023-06-23T18:35:04.004 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-06-23T18:35:04.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-06-23T18:35:04.209 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-06-23T18:35:04.277 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T18:35:04.277 INFO:tasks.ceph:role: osd.3 2023-06-23T18:35:04.277 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:35:04.278 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-23T18:35:04.351 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:35:04.351 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:35:04.352 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:35:04.352 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-23T18:35:04.352 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:35:04.352 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:35:04.352 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:35:04.352 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:35:04.352 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:35:04.353 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:35:04.791 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2023-06-23T18:35:04.793 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2023-06-23T18:35:04.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-06-23T18:35:04.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-06-23T18:35:04.999 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:35:05.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:05.225+0000 7f35f8f53640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-06-23T18:35:05.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:05.225+0000 7f35f8f53640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-06-23T18:35:05.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:05.225+0000 7f35f8f53640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-23T18:35:05.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:05.225+0000 7f35f8f53640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-23T18:35:05.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:05.225+0000 7f35f8f53640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-23T18:35:05.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:05.225+0000 7f35f8f53640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-06-23T18:35:08.107 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-06-23T18:35:08.138 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:35:08.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:08.226+0000 7f6d60db5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-06-23T18:35:08.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:08.226+0000 7f6d60db5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-06-23T18:35:08.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:08.226+0000 7f6d60db5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-23T18:35:08.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:08.226+0000 7f6d60db5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-23T18:35:08.232 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:08.226+0000 7f6d60db5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-23T18:35:08.232 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:08.226+0000 7f6d60db5640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-06-23T18:35:11.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-06-23T18:35:11.129 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:35:11.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:11.215+0000 7f0450876640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-06-23T18:35:11.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:11.215+0000 7f0450876640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-06-23T18:35:11.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:11.215+0000 7f0450876640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-23T18:35:11.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:11.215+0000 7f0450876640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-23T18:35:11.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:11.215+0000 7f0450876640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-23T18:35:11.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:11.216+0000 7f0450876640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-06-23T18:35:14.082 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-06-23T18:35:14.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:35:14.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:14.199+0000 7f2cc6cfc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-06-23T18:35:14.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:14.200+0000 7f2cc6cfc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-06-23T18:35:14.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:14.200+0000 7f2cc6cfc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-23T18:35:14.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:14.200+0000 7f2cc6cfc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-23T18:35:14.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:14.200+0000 7f2cc6cfc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-23T18:35:14.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:14.200+0000 7f2cc6cfc640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-06-23T18:35:17.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-06-23T18:35:17.098 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-06-23T18:35:17.131 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T18:35:17.132 INFO:tasks.ceph:role: osd.4 2023-06-23T18:35:17.132 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:35:17.132 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-23T18:35:17.210 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:35:17.210 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:35:17.210 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:35:17.210 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-23T18:35:17.211 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:35:17.211 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:35:17.211 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:35:17.211 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:35:17.211 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:35:17.211 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:35:17.648 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2023-06-23T18:35:17.650 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2023-06-23T18:35:17.651 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-06-23T18:35:17.815 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-06-23T18:35:17.852 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-06-23T18:35:17.922 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T18:35:17.923 INFO:tasks.ceph:role: osd.5 2023-06-23T18:35:17.923 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:35:17.924 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-23T18:35:17.999 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:35:17.999 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:35:17.999 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:35:18.000 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-23T18:35:18.000 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:35:18.000 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:35:18.000 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:35:18.000 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:35:18.000 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:35:18.000 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:35:18.419 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2023-06-23T18:35:18.420 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2023-06-23T18:35:18.421 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-06-23T18:35:18.543 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-06-23T18:35:18.577 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-06-23T18:35:18.646 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T18:35:18.647 INFO:tasks.ceph:role: osd.6 2023-06-23T18:35:18.647 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:35:18.647 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-23T18:35:18.722 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:35:18.723 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:35:18.723 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:35:18.723 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-23T18:35:18.724 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:35:18.724 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:35:18.724 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:35:18.725 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:35:18.725 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:35:18.725 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:35:19.164 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2023-06-23T18:35:19.166 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2023-06-23T18:35:19.166 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-06-23T18:35:19.316 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-06-23T18:35:19.352 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-06-23T18:35:19.421 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T18:35:19.422 INFO:tasks.ceph:role: osd.7 2023-06-23T18:35:19.422 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:35:19.422 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-23T18:35:19.496 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:35:19.496 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:35:19.496 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:35:19.496 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-23T18:35:19.497 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:35:19.497 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:35:19.497 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:35:19.497 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:35:19.497 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:35:19.497 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:35:19.942 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2023-06-23T18:35:19.944 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2023-06-23T18:35:19.944 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-06-23T18:35:20.117 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-06-23T18:35:20.150 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:35:20.265 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:20.259+0000 7f6c9d54f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-06-23T18:35:20.265 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:20.260+0000 7f6c9d54f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-06-23T18:35:20.266 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:20.260+0000 7f6c9d54f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-23T18:35:20.266 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:20.260+0000 7f6c9d54f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-23T18:35:20.266 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:20.260+0000 7f6c9d54f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-23T18:35:20.266 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:20.260+0000 7f6c9d54f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-06-23T18:35:23.136 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-06-23T18:35:23.167 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:35:23.256 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:23.251+0000 7fb3bca48640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-06-23T18:35:23.256 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:23.251+0000 7fb3bca48640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-06-23T18:35:23.257 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:23.252+0000 7fb3bca48640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-23T18:35:23.257 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:23.252+0000 7fb3bca48640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-23T18:35:23.257 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:23.252+0000 7fb3bca48640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-23T18:35:23.257 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:23.252+0000 7fb3bca48640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-06-23T18:35:26.118 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-06-23T18:35:26.147 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:35:26.236 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:26.231+0000 7fd2d6420640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-06-23T18:35:26.236 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:26.231+0000 7fd2d6420640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-06-23T18:35:26.236 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:26.231+0000 7fd2d6420640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-23T18:35:26.236 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:26.231+0000 7fd2d6420640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-23T18:35:26.236 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:26.231+0000 7fd2d6420640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-23T18:35:26.237 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:26.232+0000 7fd2d6420640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-06-23T18:35:28.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:35:28.704 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:35:29.093 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-06-23T18:35:29.120 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:35:29.208 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:29.202+0000 7f6009bc1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-06-23T18:35:29.208 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:29.203+0000 7f6009bc1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-06-23T18:35:29.208 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:29.203+0000 7f6009bc1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-23T18:35:29.208 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:29.203+0000 7f6009bc1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-23T18:35:29.208 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:29.203+0000 7f6009bc1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-23T18:35:29.208 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:29.203+0000 7f6009bc1640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-06-23T18:35:32.065 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-06-23T18:35:32.095 INFO:tasks.ceph:Reading keys from all nodes... 2023-06-23T18:35:32.096 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:32.096 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-06-23T18:35:32.127 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:32.128 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-06-23T18:35:32.199 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:32.199 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-06-23T18:35:32.269 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:32.269 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-06-23T18:35:32.338 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:32.339 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-06-23T18:35:32.408 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:32.408 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-06-23T18:35:32.476 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:32.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-06-23T18:35:32.546 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:32.547 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-06-23T18:35:32.578 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:32.579 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-06-23T18:35:32.648 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:32.649 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-06-23T18:35:32.718 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:32.718 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-06-23T18:35:32.786 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:32.786 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-06-23T18:35:32.854 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:32.854 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-06-23T18:35:32.921 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:32.922 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-06-23T18:35:32.988 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:32.988 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-06-23T18:35:33.007 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:33.008 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-06-23T18:35:33.066 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:33.066 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-06-23T18:35:33.083 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:33.084 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-06-23T18:35:33.140 INFO:tasks.ceph:Adding keys to all mons... 2023-06-23T18:35:33.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-23T18:35:33.143 DEBUG:teuthology.orchestra.run.smithi155:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-23T18:35:33.183 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.y] 2023-06-23T18:35:33.183 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDV5ZVkFO5vDRAAeqcx8oxKGghSCg1u4rWQ/A== 2023-06-23T18:35:33.183 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.a] 2023-06-23T18:35:33.184 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDV5ZVkRuZGFRAAHZmzHrtnBMKcH3vZc5GTMQ== 2023-06-23T18:35:33.184 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.c] 2023-06-23T18:35:33.184 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDV5ZVk+s6HHBAAs45yP9HZPsIUQ2b5iotFOQ== 2023-06-23T18:35:33.184 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.0] 2023-06-23T18:35:33.184 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDZ5ZVkGIZ8DRAANJGztf5I1J0TeXhbtfaR6g== 2023-06-23T18:35:33.184 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.1] 2023-06-23T18:35:33.184 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDc5ZVkNCyLDRAA3Nl41mzAhsM9zN4ZXoWavw== 2023-06-23T18:35:33.185 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.2] 2023-06-23T18:35:33.185 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDf5ZVkRBjlDBAA2jC3fNJysHoW/tMJGmyv9Q== 2023-06-23T18:35:33.185 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.3] 2023-06-23T18:35:33.185 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDi5ZVkisr5CxAArmGjhDtCQiKzqwQw29/L1w== 2023-06-23T18:35:33.185 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.x] 2023-06-23T18:35:33.185 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDV5ZVkAsHsERAAFNSGaIjAfuHkhxdQi1vpjg== 2023-06-23T18:35:33.185 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.b] 2023-06-23T18:35:33.186 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDV5ZVkKDaOIRAAU6cy09C9y6xYdKckBmHgXg== 2023-06-23T18:35:33.186 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.d] 2023-06-23T18:35:33.186 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDV5ZVkJcPYKBAAHsrS6wdO7Bo+wytNDcbHgA== 2023-06-23T18:35:33.186 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.4] 2023-06-23T18:35:33.186 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDo5ZVkQAyLDxAAekbATinpByJ83JjzbSYkOQ== 2023-06-23T18:35:33.186 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.5] 2023-06-23T18:35:33.186 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDr5ZVkFnUODxAArCgvE9RdF7FGM/U3JNYV+w== 2023-06-23T18:35:33.186 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.6] 2023-06-23T18:35:33.187 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDu5ZVkW3PWDRAAQw0817RbYvN4IBcVFyhpnQ== 2023-06-23T18:35:33.187 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.7] 2023-06-23T18:35:33.187 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDx5ZVkwfQjDBAADPpq9Rb1V8q5R1zXTBMbaw== 2023-06-23T18:35:33.187 INFO:teuthology.orchestra.run.smithi079.stdout:[client.0] 2023-06-23T18:35:33.187 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDV5ZVkbyA+LRAA4TQMvihkdeAgh/w8764lVA== 2023-06-23T18:35:33.187 INFO:teuthology.orchestra.run.smithi079.stdout:[client.2] 2023-06-23T18:35:33.187 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDV5ZVksOp9MBAAc9mALTNbDtZw8UO1fZNl9A== 2023-06-23T18:35:33.188 INFO:teuthology.orchestra.run.smithi079.stdout:[client.1] 2023-06-23T18:35:33.188 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDV5ZVkv4nPMxAAGMmXeX2s6ITjDkz5WmEAcA== 2023-06-23T18:35:33.188 INFO:teuthology.orchestra.run.smithi079.stdout:[client.3] 2023-06-23T18:35:33.188 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDV5ZVkAYOHORAA8sPnGr3zVpU78Yc8Zz1z2w== 2023-06-23T18:35:33.206 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr.y] 2023-06-23T18:35:33.207 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDV5ZVkFO5vDRAAeqcx8oxKGghSCg1u4rWQ/A== 2023-06-23T18:35:33.207 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.a] 2023-06-23T18:35:33.207 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDV5ZVkRuZGFRAAHZmzHrtnBMKcH3vZc5GTMQ== 2023-06-23T18:35:33.207 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.c] 2023-06-23T18:35:33.207 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDV5ZVk+s6HHBAAs45yP9HZPsIUQ2b5iotFOQ== 2023-06-23T18:35:33.207 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.0] 2023-06-23T18:35:33.207 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDZ5ZVkGIZ8DRAANJGztf5I1J0TeXhbtfaR6g== 2023-06-23T18:35:33.208 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.1] 2023-06-23T18:35:33.208 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDc5ZVkNCyLDRAA3Nl41mzAhsM9zN4ZXoWavw== 2023-06-23T18:35:33.208 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.2] 2023-06-23T18:35:33.208 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDf5ZVkRBjlDBAA2jC3fNJysHoW/tMJGmyv9Q== 2023-06-23T18:35:33.208 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.3] 2023-06-23T18:35:33.208 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDi5ZVkisr5CxAArmGjhDtCQiKzqwQw29/L1w== 2023-06-23T18:35:33.208 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr.x] 2023-06-23T18:35:33.209 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDV5ZVkAsHsERAAFNSGaIjAfuHkhxdQi1vpjg== 2023-06-23T18:35:33.209 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.b] 2023-06-23T18:35:33.209 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDV5ZVkKDaOIRAAU6cy09C9y6xYdKckBmHgXg== 2023-06-23T18:35:33.209 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.d] 2023-06-23T18:35:33.209 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDV5ZVkJcPYKBAAHsrS6wdO7Bo+wytNDcbHgA== 2023-06-23T18:35:33.209 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.4] 2023-06-23T18:35:33.209 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDo5ZVkQAyLDxAAekbATinpByJ83JjzbSYkOQ== 2023-06-23T18:35:33.210 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.5] 2023-06-23T18:35:33.210 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDr5ZVkFnUODxAArCgvE9RdF7FGM/U3JNYV+w== 2023-06-23T18:35:33.210 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.6] 2023-06-23T18:35:33.210 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDu5ZVkW3PWDRAAQw0817RbYvN4IBcVFyhpnQ== 2023-06-23T18:35:33.210 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.7] 2023-06-23T18:35:33.210 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDx5ZVkwfQjDBAADPpq9Rb1V8q5R1zXTBMbaw== 2023-06-23T18:35:33.210 INFO:teuthology.orchestra.run.smithi155.stdout:[client.0] 2023-06-23T18:35:33.210 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDV5ZVkbyA+LRAA4TQMvihkdeAgh/w8764lVA== 2023-06-23T18:35:33.211 INFO:teuthology.orchestra.run.smithi155.stdout:[client.2] 2023-06-23T18:35:33.211 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDV5ZVksOp9MBAAc9mALTNbDtZw8UO1fZNl9A== 2023-06-23T18:35:33.211 INFO:teuthology.orchestra.run.smithi155.stdout:[client.1] 2023-06-23T18:35:33.211 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDV5ZVkv4nPMxAAGMmXeX2s6ITjDkz5WmEAcA== 2023-06-23T18:35:33.211 INFO:teuthology.orchestra.run.smithi155.stdout:[client.3] 2023-06-23T18:35:33.211 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDV5ZVkAYOHORAA8sPnGr3zVpU78Yc8Zz1z2w== 2023-06-23T18:35:33.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T18:35:33.232 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T18:35:33.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:35:33.309 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:35:33.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:35:33.367 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:35:33.423 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.425 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.480 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.533 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.536 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.588 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.591 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T18:35:33.647 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T18:35:33.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:35:33.703 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:35:33.760 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:35:33.763 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:35:33.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.824 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.878 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.881 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.941 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.994 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:33.996 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:35:34.049 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:35:34.052 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:35:34.108 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:35:34.111 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:35:34.168 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:35:34.170 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:35:34.227 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:35:34.229 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:35:34.286 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-06-23T18:35:34.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-06-23T18:35:34.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-23T18:35:34.412 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-06-23T18:35:34.438 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-06-23T18:35:34.465 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-23T18:35:34.563 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-06-23T18:35:34.590 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-06-23T18:35:34.657 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-23T18:35:34.755 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-06-23T18:35:34.781 DEBUG:teuthology.orchestra.run.smithi079:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:35:34.783 DEBUG:teuthology.orchestra.run.smithi155:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:35:34.838 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-06-23T18:35:34.838 INFO:tasks.ceph.mon.a:Restarting daemon 2023-06-23T18:35:34.839 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-06-23T18:35:34.840 INFO:tasks.ceph.mon.a:Started 2023-06-23T18:35:34.841 INFO:tasks.ceph.mon.b:Restarting daemon 2023-06-23T18:35:34.841 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-06-23T18:35:34.880 INFO:tasks.ceph.mon.b:Started 2023-06-23T18:35:34.881 INFO:tasks.ceph.mon.c:Restarting daemon 2023-06-23T18:35:34.881 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-06-23T18:35:34.883 INFO:tasks.ceph.mon.c:Started 2023-06-23T18:35:34.883 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-06-23T18:35:34.883 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-06-23T18:35:34.884 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-06-23T18:35:34.885 INFO:tasks.ceph.mgr.y:Started 2023-06-23T18:35:34.886 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-06-23T18:35:34.886 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-06-23T18:35:34.888 INFO:tasks.ceph.mgr.x:Started 2023-06-23T18:35:34.888 INFO:tasks.ceph:Setting crush tunables to default 2023-06-23T18:35:34.888 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd crush tunables default 2023-06-23T18:35:35.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.005+0000 7f20bbd94700 1 Processor -- start 2023-06-23T18:35:35.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.005+0000 7f20bbd94700 1 -- start start 2023-06-23T18:35:35.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.006+0000 7f20bbd94700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20b41569c0 0x7f20b41608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:35.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.006+0000 7f20bbd94700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f20b406a5a0 0x7f20b40b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:35.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.006+0000 7f20bbd94700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b40cdc90 con 0x7f20b40cd440 2023-06-23T18:35:35.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.006+0000 7f20bbd94700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b40cf570 con 0x7f20b406a5a0 2023-06-23T18:35:35.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.006+0000 7f20bbd94700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b415c8b0 con 0x7f20b41569c0 2023-06-23T18:35:35.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.006+0000 7f20b932f700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f20b40cd440 0x7f20b4069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41434/0 (socket says 172.21.15.79:41434) 2023-06-23T18:35:35.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.006+0000 7f20b932f700 1 -- 172.21.15.79:0/4169998264 learned_addr learned my addr 172.21.15.79:0/4169998264 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:35.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.007+0000 7f20ba331700 1 --2- 172.21.15.79:0/4169998264 >> v2:172.21.15.155:3300/0 conn(0x7f20b406a5a0 0x7f20b40b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:35.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.007+0000 7f20b9b30700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20b41569c0 0x7f20b41608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:35.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.007+0000 7f20ba331700 1 -- 172.21.15.79:0/4169998264 >> v2:172.21.15.155:3300/0 conn(0x7f20b406a5a0 msgr2=0x7f20b40b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-23T18:35:35.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.007+0000 7f20ba331700 1 -- 172.21.15.79:0/4169998264 >> v2:172.21.15.155:3300/0 conn(0x7f20b406a5a0 msgr2=0x7f20b40b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T18:35:35.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.007+0000 7f20ba331700 1 --2- 172.21.15.79:0/4169998264 >> v2:172.21.15.155:3300/0 conn(0x7f20b406a5a0 0x7f20b40b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-23T18:35:35.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.007+0000 7f20ba331700 1 --2- 172.21.15.79:0/4169998264 >> v2:172.21.15.155:3300/0 conn(0x7f20b406a5a0 0x7f20b40b12f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T18:35:35.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.007+0000 7f20b9b30700 1 -- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20b41569c0 msgr2=0x7f20b41608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-23T18:35:35.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.007+0000 7f20b9b30700 1 -- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20b41569c0 msgr2=0x7f20b41608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T18:35:35.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.007+0000 7f20b9b30700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20b41569c0 0x7f20b41608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-23T18:35:35.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.007+0000 7f20b9b30700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20b41569c0 0x7f20b41608b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T18:35:35.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 -- 172.21.15.79:0/4169998264 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535990394 0 0) 0x7f20a4001110 con 0x7f20b40cd440 2023-06-23T18:35:35.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 -- 172.21.15.79:0/4169998264 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a00035b0 con 0x7f20b40cd440 2023-06-23T18:35:35.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 -- 172.21.15.79:0/4169998264 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 774869014 0 0) 0x7f20a00035b0 con 0x7f20b40cd440 2023-06-23T18:35:35.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 -- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20b41569c0 msgr2=0x7f20b41608b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:35:35.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20b41569c0 0x7f20b41608b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 -- 172.21.15.79:0/4169998264 >> v2:172.21.15.155:3300/0 conn(0x7f20b406a5a0 msgr2=0x7f20b40b12f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:35:35.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 --2- 172.21.15.79:0/4169998264 >> v2:172.21.15.155:3300/0 conn(0x7f20b406a5a0 0x7f20b40b12f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 -- 172.21.15.79:0/4169998264 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b415d0f0 con 0x7f20b40cd440 2023-06-23T18:35:35.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 -- 172.21.15.79:0/4169998264 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f20a4001800 con 0x7f20b40cd440 2023-06-23T18:35:35.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 -- 172.21.15.79:0/4169998264 >> v1:172.21.15.79:6789/0 conn(0x7f20b40cd440 legacy=0x7f20b4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:35.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20b41569c0 0x7f20a0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:35.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20a0004030 0x7f20a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:35.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20a0004d90 0x7f20a0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:35.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 -- 172.21.15.79:0/4169998264 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a000bac0 con 0x7f20a0004030 2023-06-23T18:35:35.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 -- 172.21.15.79:0/4169998264 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a000bc00 con 0x7f20a0004d90 2023-06-23T18:35:35.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.202+0000 7f20b8b2e700 1 -- 172.21.15.79:0/4169998264 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a000bd90 con 0x7f20b41569c0 2023-06-23T18:35:35.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.203+0000 7f20b932f700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20a0004030 0x7f20a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:35.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.203+0000 7f20b9b30700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20b41569c0 0x7f20a0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:35.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.203+0000 7f20ba331700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20a0004d90 0x7f20a0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:35.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.203+0000 7f20b932f700 1 -- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20b41569c0 msgr2=0x7f20a0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:35.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.203+0000 7f20b932f700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20b41569c0 0x7f20a0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.203+0000 7f20b932f700 1 -- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20a0004d90 msgr2=0x7f20a0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:35.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.203+0000 7f20b932f700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20a0004d90 0x7f20a0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.203+0000 7f20b932f700 1 -- 172.21.15.79:0/4169998264 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a00035b0 con 0x7f20a0004030 2023-06-23T18:35:35.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.203+0000 7f20b9b30700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20b41569c0 0x7f20a0003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:35:35.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.203+0000 7f20b932f700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20a0004030 0x7f20a0004850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f20a400c2f0 tx=0x7f20a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37fa6e0b54e255a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:35.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.203+0000 7f20b8b2e700 1 -- 172.21.15.79:0/4169998264 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a400cdf0 con 0x7f20a0004030 2023-06-23T18:35:35.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.203+0000 7f20b8b2e700 1 -- 172.21.15.79:0/4169998264 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a4005960 con 0x7f20a0004030 2023-06-23T18:35:35.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.203+0000 7f20b8b2e700 1 -- 172.21.15.79:0/4169998264 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4005d10 con 0x7f20a0004030 2023-06-23T18:35:35.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.204+0000 7f20bbd94700 1 -- 172.21.15.79:0/4169998264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20a0004030 msgr2=0x7f20a0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:35.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.204+0000 7f20bbd94700 1 --2- 172.21.15.79:0/4169998264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20a0004030 0x7f20a0004850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f20a400c2f0 tx=0x7f20a0005880 comp rx=0 tx=0).stop 2023-06-23T18:35:35.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.204+0000 7f20ba331700 1 -- 172.21.15.79:0/4169998264 reap_dead start 2023-06-23T18:35:35.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.204+0000 7f20bbd94700 1 -- 172.21.15.79:0/4169998264 shutdown_connections 2023-06-23T18:35:35.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.204+0000 7f20bbd94700 1 -- 172.21.15.79:0/4169998264 >> 172.21.15.79:0/4169998264 conn(0x7f20b406afa0 msgr2=0x7f20b406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:35.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.204+0000 7f20bbd94700 1 -- 172.21.15.79:0/4169998264 shutdown_connections 2023-06-23T18:35:35.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.204+0000 7f20bbd94700 1 -- 172.21.15.79:0/4169998264 wait complete. 2023-06-23T18:35:35.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.205+0000 7f20bbd94700 1 Processor -- start 2023-06-23T18:35:35.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.205+0000 7f20bbd94700 1 -- start start 2023-06-23T18:35:35.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20bbd94700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20a0004030 0x7f20b4155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:35.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20bbd94700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b40b1140 0x7f20b40b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:35.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20bbd94700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b40b3540 0x7f20b40b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:35.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20bbd94700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40b1af0 con 0x7f20b40b1140 2023-06-23T18:35:35.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20bbd94700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40ab8b0 con 0x7f20b40b3540 2023-06-23T18:35:35.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20bbd94700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40af0d0 con 0x7f20a0004030 2023-06-23T18:35:35.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20b932f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b40b1140 0x7f20b40b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:35.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20b9b30700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20a0004030 0x7f20b4155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:35.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20b932f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b40b1140 0x7f20b40b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38418/0 (socket says 172.21.15.79:38418) 2023-06-23T18:35:35.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20b9b30700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20a0004030 0x7f20b4155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:59416/0 (socket says 172.21.15.79:59416) 2023-06-23T18:35:35.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20b932f700 1 -- 172.21.15.79:0/3065452641 learned_addr learned my addr 172.21.15.79:0/3065452641 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:35.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20ba331700 1 --2- 172.21.15.79:0/3065452641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b40b3540 0x7f20b40b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:35.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20b932f700 1 -- 172.21.15.79:0/3065452641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20a0004030 msgr2=0x7f20b4155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:35.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20b932f700 1 --2- 172.21.15.79:0/3065452641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20a0004030 0x7f20b4155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20b932f700 1 -- 172.21.15.79:0/3065452641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b40b3540 msgr2=0x7f20b40b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:35.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20b932f700 1 --2- 172.21.15.79:0/3065452641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b40b3540 0x7f20b40b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.206+0000 7f20b932f700 1 -- 172.21.15.79:0/3065452641 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b40b4330 con 0x7f20b40b1140 2023-06-23T18:35:35.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.207+0000 7f20b932f700 1 --2- 172.21.15.79:0/3065452641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b40b1140 0x7f20b40b15b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f20a400c8e0 tx=0x7f20a4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:35.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.207+0000 7f20aa7fc700 1 -- 172.21.15.79:0/3065452641 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4007d80 con 0x7f20b40b1140 2023-06-23T18:35:35.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.207+0000 7f20bbd94700 1 -- 172.21.15.79:0/3065452641 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b40b4610 con 0x7f20b40b1140 2023-06-23T18:35:35.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.207+0000 7f20aa7fc700 1 -- 172.21.15.79:0/3065452641 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a4004870 con 0x7f20b40b1140 2023-06-23T18:35:35.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.207+0000 7f20aa7fc700 1 -- 172.21.15.79:0/3065452641 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4004be0 con 0x7f20b40b1140 2023-06-23T18:35:35.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.207+0000 7f20bbd94700 1 -- 172.21.15.79:0/3065452641 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b40af320 con 0x7f20b40b1140 2023-06-23T18:35:35.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.207+0000 7f20aa7fc700 1 -- 172.21.15.79:0/3065452641 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f20a4005340 con 0x7f20b40b1140 2023-06-23T18:35:35.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.207+0000 7f20aa7fc700 1 -- 172.21.15.79:0/3065452641 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f20a40077d0 con 0x7f20b40b1140 2023-06-23T18:35:35.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.208+0000 7f20bbd94700 1 -- 172.21.15.79:0/3065452641 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b40af4e0 con 0x7f20b40b1140 2023-06-23T18:35:35.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.211+0000 7f20aa7fc700 1 -- 172.21.15.79:0/3065452641 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f20a4025070 con 0x7f20b40b1140 2023-06-23T18:35:35.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.288+0000 7f20bbd94700 1 -- 172.21.15.79:0/3065452641 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f20b40af690 con 0x7f20b40b1140 2023-06-23T18:35:35.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.290+0000 7f20aa7fc700 1 -- 172.21.15.79:0/3065452641 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f20a4017070 con 0x7f20b40b1140 2023-06-23T18:35:35.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.291+0000 7f20bbd94700 1 -- 172.21.15.79:0/3065452641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b40b1140 msgr2=0x7f20b40b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:35.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.291+0000 7f20bbd94700 1 --2- 172.21.15.79:0/3065452641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b40b1140 0x7f20b40b15b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f20a400c8e0 tx=0x7f20a4000fe0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.291+0000 7f20bbd94700 1 -- 172.21.15.79:0/3065452641 shutdown_connections 2023-06-23T18:35:35.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.291+0000 7f20bbd94700 1 --2- 172.21.15.79:0/3065452641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b40b3540 0x7f20b40b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.291+0000 7f20bbd94700 1 --2- 172.21.15.79:0/3065452641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20a0004030 0x7f20b4155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.291+0000 7f20bbd94700 1 --2- 172.21.15.79:0/3065452641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b40b1140 0x7f20b40b15b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.291+0000 7f20bbd94700 1 -- 172.21.15.79:0/3065452641 >> 172.21.15.79:0/3065452641 conn(0x7f20b406afa0 msgr2=0x7f20b4071370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:35.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.291+0000 7f20bbd94700 1 -- 172.21.15.79:0/3065452641 shutdown_connections 2023-06-23T18:35:35.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:35.291+0000 7f20bbd94700 1 -- 172.21.15.79:0/3065452641 wait complete. 2023-06-23T18:35:35.297 INFO:teuthology.orchestra.run.smithi079.stderr:adjusted tunables profile to default 2023-06-23T18:35:35.308 INFO:tasks.ceph:check_enable_crimson: False 2023-06-23T18:35:35.309 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-06-23T18:35:35.309 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:35.310 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-06-23T18:35:35.331 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:35.326+0000 7fb8a0753200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T18:35:35.334 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:35.330+0000 7f4a314e2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T18:35:35.335 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:35.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-06-23T18:35:35.395 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:35.391+0000 7fb8a0753200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T18:35:35.402 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:35.402 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-06-23T18:35:35.403 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:35.398+0000 7f4a314e2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T18:35:35.429 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:35:35.429 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-06-23T18:35:35.471 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:35.467+0000 7fb8a0753200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T18:35:35.482 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:35.477+0000 7f4a314e2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T18:35:35.495 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:35.495 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-06-23T18:35:35.523 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:35.524 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-06-23T18:35:35.555 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:35.550+0000 7fb8a0753200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T18:35:35.564 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:35.559+0000 7f4a314e2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T18:35:35.580 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:35.581 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-06-23T18:35:35.646 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:35:35.647 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-06-23T18:35:35.704 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:35.699+0000 7fb8a0753200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T18:35:35.705 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:35.701+0000 7f4a314e2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T18:35:35.712 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new e2932a84-07f5-4a23-9b19-7fd0e288a1ef 0 2023-06-23T18:35:35.874 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.869+0000 7ff6518a0700 1 Processor -- start 2023-06-23T18:35:35.874 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.869+0000 7ff6518a0700 1 -- start start 2023-06-23T18:35:35.876 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.870+0000 7ff6518a0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff64c15c2a0 0x7ff64c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:35.876 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.870+0000 7ff6518a0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff64c0b1410 0x7ff64c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:35.876 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.870+0000 7ff6518a0700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64c15ffe0 con 0x7ff64c06a3d0 2023-06-23T18:35:35.877 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.870+0000 7ff6518a0700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64c06a850 con 0x7ff64c15c2a0 2023-06-23T18:35:35.877 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.870+0000 7ff6518a0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c1602c0 con 0x7ff64c0b1410 2023-06-23T18:35:35.877 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.870+0000 7ff64b7fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7ff64c06a3d0 0x7ff64c070af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:57080/0 (socket says 172.21.15.155:57080) 2023-06-23T18:35:35.877 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.870+0000 7ff64a7fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff64c0b1410 0x7ff64c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:35.877 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.870+0000 7ff64b7fe700 1 -- 172.21.15.155:0/3035645948 learned_addr learned my addr 172.21.15.155:0/3035645948 (peer_addr_for_me v1:172.21.15.155:0/0) 2023-06-23T18:35:35.877 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.870+0000 7ff64affd700 1 --2- 172.21.15.155:0/3035645948 >> v2:172.21.15.79:3300/0 conn(0x7ff64c15c2a0 0x7ff64c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:35.877 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.870+0000 7ff649ffb700 1 -- 172.21.15.155:0/3035645948 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269661740 0 0) 0x7ff64c15ffe0 con 0x7ff64c06a3d0 2023-06-23T18:35:35.877 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.870+0000 7ff649ffb700 1 -- 172.21.15.155:0/3035645948 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6340035b0 con 0x7ff64c06a3d0 2023-06-23T18:35:35.878 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.871+0000 7ff64a7fc700 1 -- 172.21.15.155:0/3035645948 >> v2:172.21.15.79:3300/0 conn(0x7ff64c15c2a0 msgr2=0x7ff64c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:35.878 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.871+0000 7ff64a7fc700 1 --2- 172.21.15.155:0/3035645948 >> v2:172.21.15.79:3300/0 conn(0x7ff64c15c2a0 0x7ff64c0ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.878 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.871+0000 7ff64a7fc700 1 -- 172.21.15.155:0/3035645948 >> v1:172.21.15.155:6789/0 conn(0x7ff64c06a3d0 legacy=0x7ff64c070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:35.878 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.871+0000 7ff64a7fc700 1 -- 172.21.15.155:0/3035645948 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c1605a0 con 0x7ff64c0b1410 2023-06-23T18:35:35.878 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.871+0000 7ff64a7fc700 1 --2- 172.21.15.155:0/3035645948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff64c0b1410 0x7ff64c069e90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff640009840 tx=0x7ff640009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30b2d4079631d5eb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:35.878 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.871+0000 7ff649ffb700 1 -- 172.21.15.155:0/3035645948 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640011830 con 0x7ff64c0b1410 2023-06-23T18:35:35.878 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.871+0000 7ff649ffb700 1 -- 172.21.15.155:0/3035645948 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff640011990 con 0x7ff64c0b1410 2023-06-23T18:35:35.879 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.871+0000 7ff649ffb700 1 -- 172.21.15.155:0/3035645948 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640011d00 con 0x7ff64c0b1410 2023-06-23T18:35:35.879 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.871+0000 7ff6518a0700 1 -- 172.21.15.155:0/3035645948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff64c0b1410 msgr2=0x7ff64c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:35.879 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.871+0000 7ff6518a0700 1 --2- 172.21.15.155:0/3035645948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff64c0b1410 0x7ff64c069e90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff640009840 tx=0x7ff640009b50 comp rx=0 tx=0).stop 2023-06-23T18:35:35.879 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.872+0000 7ff6518a0700 1 -- 172.21.15.155:0/3035645948 shutdown_connections 2023-06-23T18:35:35.879 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.872+0000 7ff6518a0700 1 --2- 172.21.15.155:0/3035645948 >> v2:172.21.15.79:3300/0 conn(0x7ff64c15c2a0 0x7ff64c0ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.879 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.872+0000 7ff6518a0700 1 --2- 172.21.15.155:0/3035645948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff64c0b1410 0x7ff64c069e90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.879 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.872+0000 7ff6518a0700 1 -- 172.21.15.155:0/3035645948 >> 172.21.15.155:0/3035645948 conn(0x7ff64c0cfc60 msgr2=0x7ff64c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:35.879 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.872+0000 7ff6518a0700 1 -- 172.21.15.155:0/3035645948 shutdown_connections 2023-06-23T18:35:35.880 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.872+0000 7ff6518a0700 1 -- 172.21.15.155:0/3035645948 wait complete. 2023-06-23T18:35:35.881 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.873+0000 7ff6518a0700 1 Processor -- start 2023-06-23T18:35:35.882 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.873+0000 7ff6518a0700 1 -- start start 2023-06-23T18:35:35.882 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff6518a0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff64c06a3d0 0x7ff64c0ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:35.882 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff6518a0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff64c0b1410 0x7ff64c0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:35.882 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff6518a0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff64c15c2a0 0x7ff64c0b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:35.882 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff6518a0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0c2b80 con 0x7ff64c06a3d0 2023-06-23T18:35:35.882 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff6518a0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0c2e60 con 0x7ff64c15c2a0 2023-06-23T18:35:35.882 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff6518a0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0c3140 con 0x7ff64c0b1410 2023-06-23T18:35:35.883 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff64a7fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff64c0b1410 0x7ff64c0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:35.883 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff64b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff64c15c2a0 0x7ff64c0b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:35.883 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff64a7fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff64c0b1410 0x7ff64c0add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:56142/0 (socket says 172.21.15.155:56142) 2023-06-23T18:35:35.883 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff64a7fc700 1 -- 172.21.15.155:0/2212149907 learned_addr learned my addr 172.21.15.155:0/2212149907 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-06-23T18:35:35.883 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff64affd700 1 --2- 172.21.15.155:0/2212149907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff64c06a3d0 0x7ff64c0ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:35.883 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff64a7fc700 1 -- 172.21.15.155:0/2212149907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff64c15c2a0 msgr2=0x7ff64c0b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:35.883 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff64a7fc700 1 --2- 172.21.15.155:0/2212149907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff64c15c2a0 0x7ff64c0b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.884 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff64a7fc700 1 -- 172.21.15.155:0/2212149907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff64c06a3d0 msgr2=0x7ff64c0ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:35.884 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff64a7fc700 1 --2- 172.21.15.155:0/2212149907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff64c06a3d0 0x7ff64c0ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.884 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff64a7fc700 1 -- 172.21.15.155:0/2212149907 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6400094a0 con 0x7ff64c0b1410 2023-06-23T18:35:35.884 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff64b7fe700 1 --2- 172.21.15.155:0/2212149907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff64c15c2a0 0x7ff64c0b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:35.884 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff64affd700 1 --2- 172.21.15.155:0/2212149907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff64c06a3d0 0x7ff64c0ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:35.884 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff64a7fc700 1 --2- 172.21.15.155:0/2212149907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff64c0b1410 0x7ff64c0add40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff640009fd0 tx=0x7ff640004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:35.884 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff633fff700 1 -- 172.21.15.155:0/2212149907 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640004550 con 0x7ff64c0b1410 2023-06-23T18:35:35.885 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff633fff700 1 -- 172.21.15.155:0/2212149907 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64001e070 con 0x7ff64c0b1410 2023-06-23T18:35:35.885 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.874+0000 7ff6518a0700 1 -- 172.21.15.155:0/2212149907 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64c0c3420 con 0x7ff64c0b1410 2023-06-23T18:35:35.885 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.875+0000 7ff633fff700 1 -- 172.21.15.155:0/2212149907 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6400038e0 con 0x7ff64c0b1410 2023-06-23T18:35:35.885 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.875+0000 7ff6518a0700 1 -- 172.21.15.155:0/2212149907 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64c0b4d80 con 0x7ff64c0b1410 2023-06-23T18:35:35.885 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.875+0000 7ff633fff700 1 -- 172.21.15.155:0/2212149907 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff640003d40 con 0x7ff64c0b1410 2023-06-23T18:35:35.885 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.875+0000 7ff633fff700 1 -- 172.21.15.155:0/2212149907 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff64002dd90 con 0x7ff64c0b1410 2023-06-23T18:35:35.885 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.875+0000 7ff6518a0700 1 -- 172.21.15.155:0/2212149907 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64c0672c0 con 0x7ff64c0b1410 2023-06-23T18:35:35.886 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.879+0000 7ff633fff700 1 -- 172.21.15.155:0/2212149907 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ff640020070 con 0x7ff64c0b1410 2023-06-23T18:35:35.966 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.961+0000 7ff6518a0700 1 -- 172.21.15.155:0/2212149907 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e2932a84-07f5-4a23-9b19-7fd0e288a1ef", "id": 0} v 0) v1 -- 0x7ff64c0ce930 con 0x7ff64c0b1410 2023-06-23T18:35:35.971 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.966+0000 7ff633fff700 1 -- 172.21.15.155:0/2212149907 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e2932a84-07f5-4a23-9b19-7fd0e288a1ef", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff640015020 con 0x7ff64c0b1410 2023-06-23T18:35:35.971 INFO:teuthology.orchestra.run.smithi155.stdout:0 2023-06-23T18:35:35.972 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.967+0000 7ff6518a0700 1 -- 172.21.15.155:0/2212149907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff64c0b1410 msgr2=0x7ff64c0add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:35.972 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.967+0000 7ff6518a0700 1 --2- 172.21.15.155:0/2212149907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff64c0b1410 0x7ff64c0add40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff640009fd0 tx=0x7ff640004750 comp rx=0 tx=0).stop 2023-06-23T18:35:35.973 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.967+0000 7ff6518a0700 1 -- 172.21.15.155:0/2212149907 shutdown_connections 2023-06-23T18:35:35.973 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.967+0000 7ff6518a0700 1 --2- 172.21.15.155:0/2212149907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff64c15c2a0 0x7ff64c0b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.973 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.967+0000 7ff6518a0700 1 --2- 172.21.15.155:0/2212149907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff64c06a3d0 0x7ff64c0ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.973 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.967+0000 7ff6518a0700 1 --2- 172.21.15.155:0/2212149907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff64c0b1410 0x7ff64c0add40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:35.973 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.967+0000 7ff6518a0700 1 -- 172.21.15.155:0/2212149907 >> 172.21.15.155:0/2212149907 conn(0x7ff64c0cfc60 msgr2=0x7ff64c0c9720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:35.973 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.968+0000 7ff6518a0700 1 -- 172.21.15.155:0/2212149907 shutdown_connections 2023-06-23T18:35:35.974 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:35.968+0000 7ff6518a0700 1 -- 172.21.15.155:0/2212149907 wait complete. 2023-06-23T18:35:35.984 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new e6b1bc10-2bc9-4af2-9833-fade6110194d 1 2023-06-23T18:35:36.040 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:36.035+0000 7f4a314e2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T18:35:36.051 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:36.046+0000 7fb8a0753200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T18:35:36.105 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.100+0000 7fd3399f3700 1 Processor -- start 2023-06-23T18:35:36.105 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.100+0000 7fd3399f3700 1 -- start start 2023-06-23T18:35:36.112 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd3399f3700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd334156ae0 0x7fd33415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.112 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd3399f3700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd3340cc060 0x7fd3340cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.113 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd3399f3700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd33415daf0 con 0x7fd334071710 2023-06-23T18:35:36.113 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd3399f3700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3340cca10 con 0x7fd3340cc060 2023-06-23T18:35:36.113 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd3399f3700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33415a560 con 0x7fd334156ae0 2023-06-23T18:35:36.113 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd332ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd334156ae0 0x7fd33415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.114 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd3327fc700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd3340cc060 0x7fd3340cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.114 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd332ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd334156ae0 0x7fd33415d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:56146/0 (socket says 172.21.15.155:56146) 2023-06-23T18:35:36.114 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd332ffd700 1 -- 172.21.15.155:0/1771896543 learned_addr learned my addr 172.21.15.155:0/1771896543 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-06-23T18:35:36.114 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd3327fc700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd3340cc060 0x7fd3340cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40358/0 (socket says 172.21.15.155:40358) 2023-06-23T18:35:36.115 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd332ffd700 1 -- 172.21.15.155:0/1771896543 >> v2:172.21.15.155:3300/0 conn(0x7fd3340cc060 msgr2=0x7fd3340cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:36.115 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd332ffd700 1 --2- 172.21.15.155:0/1771896543 >> v2:172.21.15.155:3300/0 conn(0x7fd3340cc060 0x7fd3340cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.115 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd332ffd700 1 -- 172.21.15.155:0/1771896543 >> v1:172.21.15.79:6789/0 conn(0x7fd334071710 legacy=0x7fd334071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.115 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd332ffd700 1 -- 172.21.15.155:0/1771896543 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd334159950 con 0x7fd334156ae0 2023-06-23T18:35:36.115 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd3327fc700 1 --2- 172.21.15.155:0/1771896543 >> v2:172.21.15.155:3300/0 conn(0x7fd3340cc060 0x7fd3340cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.101+0000 7fd332ffd700 1 --2- 172.21.15.155:0/1771896543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd334156ae0 0x7fd33415d5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd32400bb90 tx=0x7fd32400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=333acc15e56f7b9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.102+0000 7fd331ffb700 1 -- 172.21.15.155:0/1771896543 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32400dd30 con 0x7fd334156ae0 2023-06-23T18:35:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.102+0000 7fd331ffb700 1 -- 172.21.15.155:0/1771896543 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32400de90 con 0x7fd334156ae0 2023-06-23T18:35:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.102+0000 7fd331ffb700 1 -- 172.21.15.155:0/1771896543 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324010690 con 0x7fd334156ae0 2023-06-23T18:35:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.102+0000 7fd3399f3700 1 -- 172.21.15.155:0/1771896543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd334156ae0 msgr2=0x7fd33415d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.116 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.102+0000 7fd3399f3700 1 --2- 172.21.15.155:0/1771896543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd334156ae0 0x7fd33415d5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd32400bb90 tx=0x7fd32400bea0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.102+0000 7fd3399f3700 1 -- 172.21.15.155:0/1771896543 shutdown_connections 2023-06-23T18:35:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.102+0000 7fd3399f3700 1 --2- 172.21.15.155:0/1771896543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd334156ae0 0x7fd33415d5b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.102+0000 7fd3399f3700 1 --2- 172.21.15.155:0/1771896543 >> v2:172.21.15.155:3300/0 conn(0x7fd3340cc060 0x7fd3340cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.102+0000 7fd3399f3700 1 -- 172.21.15.155:0/1771896543 >> 172.21.15.155:0/1771896543 conn(0x7fd33406afa0 msgr2=0x7fd33406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.102+0000 7fd3399f3700 1 -- 172.21.15.155:0/1771896543 shutdown_connections 2023-06-23T18:35:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.102+0000 7fd3399f3700 1 -- 172.21.15.155:0/1771896543 wait complete. 2023-06-23T18:35:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.103+0000 7fd3399f3700 1 Processor -- start 2023-06-23T18:35:36.117 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.104+0000 7fd3399f3700 1 -- start start 2023-06-23T18:35:36.118 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.104+0000 7fd3399f3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd334071710 0x7fd33414bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.118 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.104+0000 7fd3399f3700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3340cc060 0x7fd3340ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.118 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.104+0000 7fd3399f3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd334156ae0 0x7fd3340abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.118 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.104+0000 7fd3399f3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3341542c0 con 0x7fd334156ae0 2023-06-23T18:35:36.118 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.104+0000 7fd3399f3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3341545a0 con 0x7fd334071710 2023-06-23T18:35:36.118 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.104+0000 7fd3399f3700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd334154880 con 0x7fd3340cc060 2023-06-23T18:35:36.118 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.104+0000 7fd332ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd334071710 0x7fd33414bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.119 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.104+0000 7fd3327fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3340cc060 0x7fd3340ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.119 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.104+0000 7fd332ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd334071710 0x7fd33414bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40362/0 (socket says 172.21.15.155:40362) 2023-06-23T18:35:36.119 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.104+0000 7fd332ffd700 1 -- 172.21.15.155:0/4156885640 learned_addr learned my addr 172.21.15.155:0/4156885640 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-06-23T18:35:36.119 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.105+0000 7fd3337fe700 1 --2- 172.21.15.155:0/4156885640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd334156ae0 0x7fd3340abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.119 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.105+0000 7fd332ffd700 1 -- 172.21.15.155:0/4156885640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3340cc060 msgr2=0x7fd3340ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.119 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.105+0000 7fd332ffd700 1 --2- 172.21.15.155:0/4156885640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3340cc060 0x7fd3340ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.119 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.105+0000 7fd332ffd700 1 -- 172.21.15.155:0/4156885640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd334156ae0 msgr2=0x7fd3340abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.120 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.105+0000 7fd332ffd700 1 --2- 172.21.15.155:0/4156885640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd334156ae0 0x7fd3340abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.120 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.105+0000 7fd332ffd700 1 -- 172.21.15.155:0/4156885640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3280094a0 con 0x7fd334071710 2023-06-23T18:35:36.120 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.105+0000 7fd332ffd700 1 --2- 172.21.15.155:0/4156885640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd334071710 0x7fd33414bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd324000c00 tx=0x7fd324004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:36.120 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.105+0000 7fd31bfff700 1 -- 172.21.15.155:0/4156885640 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324003b70 con 0x7fd334071710 2023-06-23T18:35:36.121 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.105+0000 7fd31bfff700 1 -- 172.21.15.155:0/4156885640 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd324003cd0 con 0x7fd334071710 2023-06-23T18:35:36.121 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.105+0000 7fd3399f3700 1 -- 172.21.15.155:0/4156885640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32400b7f0 con 0x7fd334071710 2023-06-23T18:35:36.121 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.105+0000 7fd31bfff700 1 -- 172.21.15.155:0/4156885640 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3240106b0 con 0x7fd334071710 2023-06-23T18:35:36.121 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.105+0000 7fd3399f3700 1 -- 172.21.15.155:0/4156885640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd334154e40 con 0x7fd334071710 2023-06-23T18:35:36.121 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.105+0000 7fd31bfff700 1 -- 172.21.15.155:0/4156885640 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd324010b10 con 0x7fd334071710 2023-06-23T18:35:36.121 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.105+0000 7fd31bfff700 1 -- 172.21.15.155:0/4156885640 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fd324019e20 con 0x7fd334071710 2023-06-23T18:35:36.122 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.106+0000 7fd3399f3700 1 -- 172.21.15.155:0/4156885640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3340672c0 con 0x7fd334071710 2023-06-23T18:35:36.122 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:36.110+0000 7f4a314e2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T18:35:36.123 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.108+0000 7fd31bfff700 1 -- 172.21.15.155:0/4156885640 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fd324003e40 con 0x7fd334071710 2023-06-23T18:35:36.130 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:36.126+0000 7fb8a0753200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T18:35:36.196 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.192+0000 7fd3399f3700 1 -- 172.21.15.155:0/4156885640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e6b1bc10-2bc9-4af2-9833-fade6110194d", "id": 1} v 0) v1 -- 0x7fd3340b1be0 con 0x7fd334071710 2023-06-23T18:35:36.202 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.198+0000 7fd31bfff700 1 -- 172.21.15.155:0/4156885640 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e6b1bc10-2bc9-4af2-9833-fade6110194d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd3240194d0 con 0x7fd334071710 2023-06-23T18:35:36.203 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-06-23T18:35:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.199+0000 7fd3399f3700 1 -- 172.21.15.155:0/4156885640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd334071710 msgr2=0x7fd33414bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.199+0000 7fd3399f3700 1 --2- 172.21.15.155:0/4156885640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd334071710 0x7fd33414bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd324000c00 tx=0x7fd324004750 comp rx=0 tx=0).stop 2023-06-23T18:35:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.199+0000 7fd3399f3700 1 -- 172.21.15.155:0/4156885640 shutdown_connections 2023-06-23T18:35:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.199+0000 7fd3399f3700 1 --2- 172.21.15.155:0/4156885640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd334156ae0 0x7fd3340abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.199+0000 7fd3399f3700 1 --2- 172.21.15.155:0/4156885640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd334071710 0x7fd33414bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.199+0000 7fd3399f3700 1 --2- 172.21.15.155:0/4156885640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3340cc060 0x7fd3340ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.205 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.199+0000 7fd3399f3700 1 -- 172.21.15.155:0/4156885640 >> 172.21.15.155:0/4156885640 conn(0x7fd33406afa0 msgr2=0x7fd334155c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:36.205 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.199+0000 7fd3399f3700 1 -- 172.21.15.155:0/4156885640 shutdown_connections 2023-06-23T18:35:36.205 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.199+0000 7fd3399f3700 1 -- 172.21.15.155:0/4156885640 wait complete. 2023-06-23T18:35:36.216 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 5b16c204-3aa9-4741-89d4-f8c1530b399b 2 2023-06-23T18:35:36.243 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:36.238+0000 7f4a314e2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T18:35:36.260 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:36.255+0000 7fb8a0753200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T18:35:36.323 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:36.318+0000 7f4a314e2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T18:35:36.339 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.334+0000 7fcb855f9700 1 Processor -- start 2023-06-23T18:35:36.339 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.334+0000 7fcb855f9700 1 -- start start 2023-06-23T18:35:36.340 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.335+0000 7fcb855f9700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcb800cd440 0x7fcb801608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.340 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.335+0000 7fcb855f9700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb800cdc90 con 0x7fcb8006a5a0 2023-06-23T18:35:36.340 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.335+0000 7fcb855f9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8015d0f0 con 0x7fcb801569c0 2023-06-23T18:35:36.341 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.335+0000 7fcb855f9700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb800cf570 con 0x7fcb800cd440 2023-06-23T18:35:36.341 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.335+0000 7fcb7f7fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcb800cd440 0x7fcb801608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.341 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.335+0000 7fcb7effd700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fcb801569c0 0x7fcb80069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:57100/0 (socket says 172.21.15.155:57100) 2023-06-23T18:35:36.341 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.335+0000 7fcb7effd700 1 -- 172.21.15.155:0/1706649764 learned_addr learned my addr 172.21.15.155:0/1706649764 (peer_addr_for_me v1:172.21.15.155:0/0) 2023-06-23T18:35:36.341 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.335+0000 7fcb7f7fe700 1 -- 172.21.15.155:0/1706649764 >> v1:172.21.15.155:6789/0 conn(0x7fcb801569c0 legacy=0x7fcb80069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.341 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.335+0000 7fcb7f7fe700 1 -- 172.21.15.155:0/1706649764 >> v1:172.21.15.79:6789/0 conn(0x7fcb8006a5a0 legacy=0x7fcb800b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.341 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.335+0000 7fcb7f7fe700 1 -- 172.21.15.155:0/1706649764 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb80156d90 con 0x7fcb800cd440 2023-06-23T18:35:36.342 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.336+0000 7fcb7f7fe700 1 --2- 172.21.15.155:0/1706649764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb800cd440 0x7fcb801608b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009d90 tx=0x7fcb7800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70b757d766f466a8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:36.342 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.336+0000 7fcb7dffb700 1 -- 172.21.15.155:0/1706649764 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7800e070 con 0x7fcb800cd440 2023-06-23T18:35:36.342 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.336+0000 7fcb7dffb700 1 -- 172.21.15.155:0/1706649764 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7800bbd0 con 0x7fcb800cd440 2023-06-23T18:35:36.342 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.336+0000 7fcb7dffb700 1 -- 172.21.15.155:0/1706649764 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78012430 con 0x7fcb800cd440 2023-06-23T18:35:36.344 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.336+0000 7fcb855f9700 1 -- 172.21.15.155:0/1706649764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb800cd440 msgr2=0x7fcb801608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.344 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.336+0000 7fcb855f9700 1 --2- 172.21.15.155:0/1706649764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb800cd440 0x7fcb801608b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009d90 tx=0x7fcb7800b330 comp rx=0 tx=0).stop 2023-06-23T18:35:36.344 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.336+0000 7fcb855f9700 1 -- 172.21.15.155:0/1706649764 shutdown_connections 2023-06-23T18:35:36.345 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.336+0000 7fcb855f9700 1 --2- 172.21.15.155:0/1706649764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb800cd440 0x7fcb801608b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.345 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.336+0000 7fcb855f9700 1 -- 172.21.15.155:0/1706649764 >> 172.21.15.155:0/1706649764 conn(0x7fcb8006afa0 msgr2=0x7fcb8006b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:36.345 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.336+0000 7fcb855f9700 1 -- 172.21.15.155:0/1706649764 shutdown_connections 2023-06-23T18:35:36.345 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.336+0000 7fcb855f9700 1 -- 172.21.15.155:0/1706649764 wait complete. 2023-06-23T18:35:36.345 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.338+0000 7fcb855f9700 1 Processor -- start 2023-06-23T18:35:36.345 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.338+0000 7fcb855f9700 1 -- start start 2023-06-23T18:35:36.345 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.338+0000 7fcb855f9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb8006a5a0 0x7fcb800c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.345 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.338+0000 7fcb855f9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb800cd440 0x7fcb8015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.346 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.338+0000 7fcb855f9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb801569c0 0x7fcb8015e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.346 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.338+0000 7fcb855f9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800c79c0 con 0x7fcb800cd440 2023-06-23T18:35:36.346 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.338+0000 7fcb855f9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800c7400 con 0x7fcb801569c0 2023-06-23T18:35:36.346 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.338+0000 7fcb855f9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800c76e0 con 0x7fcb8006a5a0 2023-06-23T18:35:36.346 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb7effd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb8006a5a0 0x7fcb800c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.346 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb7effd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb8006a5a0 0x7fcb800c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:56164/0 (socket says 172.21.15.155:56164) 2023-06-23T18:35:36.346 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb7effd700 1 -- 172.21.15.155:0/1003342135 learned_addr learned my addr 172.21.15.155:0/1003342135 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-06-23T18:35:36.347 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb7f7fe700 1 --2- 172.21.15.155:0/1003342135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb801569c0 0x7fcb8015e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.347 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb7e7fc700 1 --2- 172.21.15.155:0/1003342135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb800cd440 0x7fcb8015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.347 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb7f7fe700 1 -- 172.21.15.155:0/1003342135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb8006a5a0 msgr2=0x7fcb800c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.347 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb7f7fe700 1 --2- 172.21.15.155:0/1003342135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb8006a5a0 0x7fcb800c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.347 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb7f7fe700 1 -- 172.21.15.155:0/1003342135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb800cd440 msgr2=0x7fcb8015dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.347 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb7f7fe700 1 --2- 172.21.15.155:0/1003342135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb800cd440 0x7fcb8015dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.347 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb7f7fe700 1 -- 172.21.15.155:0/1003342135 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb78009a70 con 0x7fcb801569c0 2023-06-23T18:35:36.347 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb7effd700 1 --2- 172.21.15.155:0/1003342135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb8006a5a0 0x7fcb800c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:36.348 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb7f7fe700 1 --2- 172.21.15.155:0/1003342135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb801569c0 0x7fcb8015e290 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcb78016000 tx=0x7fcb78004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:36.348 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb677fe700 1 -- 172.21.15.155:0/1003342135 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78012da0 con 0x7fcb801569c0 2023-06-23T18:35:36.348 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb677fe700 1 -- 172.21.15.155:0/1003342135 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb78020070 con 0x7fcb801569c0 2023-06-23T18:35:36.348 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb855f9700 1 -- 172.21.15.155:0/1003342135 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb800c3ff0 con 0x7fcb801569c0 2023-06-23T18:35:36.348 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb677fe700 1 -- 172.21.15.155:0/1003342135 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78004280 con 0x7fcb801569c0 2023-06-23T18:35:36.348 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb855f9700 1 -- 172.21.15.155:0/1003342135 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb80150250 con 0x7fcb801569c0 2023-06-23T18:35:36.348 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.339+0000 7fcb677fe700 1 -- 172.21.15.155:0/1003342135 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcb780046e0 con 0x7fcb801569c0 2023-06-23T18:35:36.349 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.340+0000 7fcb677fe700 1 -- 172.21.15.155:0/1003342135 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fcb78022070 con 0x7fcb801569c0 2023-06-23T18:35:36.349 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:36.340+0000 7fb8a0753200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T18:35:36.349 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.340+0000 7fcb657fa700 1 -- 172.21.15.155:0/1003342135 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb580052f0 con 0x7fcb801569c0 2023-06-23T18:35:36.350 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.342+0000 7fcb677fe700 1 -- 172.21.15.155:0/1003342135 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fcb78045b00 con 0x7fcb801569c0 2023-06-23T18:35:36.431 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.426+0000 7fcb657fa700 1 -- 172.21.15.155:0/1003342135 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5b16c204-3aa9-4741-89d4-f8c1530b399b", "id": 2} v 0) v1 -- 0x7fcb58005910 con 0x7fcb801569c0 2023-06-23T18:35:36.438 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.434+0000 7fcb677fe700 1 -- 172.21.15.155:0/1003342135 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5b16c204-3aa9-4741-89d4-f8c1530b399b", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcb78034030 con 0x7fcb801569c0 2023-06-23T18:35:36.439 INFO:teuthology.orchestra.run.smithi155.stdout:2 2023-06-23T18:35:36.441 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.435+0000 7fcb657fa700 1 -- 172.21.15.155:0/1003342135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb801569c0 msgr2=0x7fcb8015e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.441 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.435+0000 7fcb657fa700 1 --2- 172.21.15.155:0/1003342135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb801569c0 0x7fcb8015e290 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcb78016000 tx=0x7fcb78004ad0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.441 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.435+0000 7fcb657fa700 1 -- 172.21.15.155:0/1003342135 shutdown_connections 2023-06-23T18:35:36.442 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.435+0000 7fcb657fa700 1 --2- 172.21.15.155:0/1003342135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb801569c0 0x7fcb8015e290 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.442 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.435+0000 7fcb657fa700 1 --2- 172.21.15.155:0/1003342135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb8006a5a0 0x7fcb800c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.442 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.435+0000 7fcb657fa700 1 --2- 172.21.15.155:0/1003342135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb800cd440 0x7fcb8015dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.443 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.435+0000 7fcb657fa700 1 -- 172.21.15.155:0/1003342135 >> 172.21.15.155:0/1003342135 conn(0x7fcb8006afa0 msgr2=0x7fcb80071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:36.443 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.435+0000 7fcb657fa700 1 -- 172.21.15.155:0/1003342135 shutdown_connections 2023-06-23T18:35:36.444 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.435+0000 7fcb657fa700 1 -- 172.21.15.155:0/1003342135 wait complete. 2023-06-23T18:35:36.451 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new d2efa121-6af5-4b4d-b2df-2df7070cc986 3 2023-06-23T18:35:36.615 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.610+0000 7f99183ec700 1 Processor -- start 2023-06-23T18:35:36.615 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.610+0000 7f99183ec700 1 -- start start 2023-06-23T18:35:36.616 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f99183ec700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f99100cedf0 0x7f99100b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.616 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f99183ec700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99101438f0 0x7f99100b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.617 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f99183ec700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99100b0b10 con 0x7f99100d08c0 2023-06-23T18:35:36.617 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f99183ec700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99100b0df0 con 0x7f99100cedf0 2023-06-23T18:35:36.617 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f99183ec700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99100b10d0 con 0x7f99101438f0 2023-06-23T18:35:36.617 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f9916188700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f99100cedf0 0x7f99100b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.617 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f9915987700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99101438f0 0x7f99100b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.617 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f9916188700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f99100cedf0 0x7f99100b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40380/0 (socket says 172.21.15.155:40380) 2023-06-23T18:35:36.617 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f9916188700 1 -- 172.21.15.155:0/308684334 learned_addr learned my addr 172.21.15.155:0/308684334 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-06-23T18:35:36.617 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f9915987700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99101438f0 0x7f99100b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:56170/0 (socket says 172.21.15.155:56170) 2023-06-23T18:35:36.618 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f9915987700 1 -- 172.21.15.155:0/308684334 >> v2:172.21.15.155:3300/0 conn(0x7f99100cedf0 msgr2=0x7f99100b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:36.618 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f9915987700 1 --2- 172.21.15.155:0/308684334 >> v2:172.21.15.155:3300/0 conn(0x7f99100cedf0 0x7f99100b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.618 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f9915987700 1 -- 172.21.15.155:0/308684334 >> v1:172.21.15.79:6789/0 conn(0x7f99100d08c0 legacy=0x7f99100ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:36.618 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f9915987700 1 -- 172.21.15.155:0/308684334 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99100b13b0 con 0x7f99101438f0 2023-06-23T18:35:36.618 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.611+0000 7f9916188700 1 --2- 172.21.15.155:0/308684334 >> v2:172.21.15.155:3300/0 conn(0x7f99100cedf0 0x7f99100b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:36.618 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.612+0000 7f9915987700 1 --2- 172.21.15.155:0/308684334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99101438f0 0x7f99100b1bb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f990c009840 tx=0x7f990c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=470faa720ec0ea71 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:36.619 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.612+0000 7f9915186700 1 -- 172.21.15.155:0/308684334 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c004230 con 0x7f99101438f0 2023-06-23T18:35:36.619 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.612+0000 7f9915186700 1 -- 172.21.15.155:0/308684334 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c004390 con 0x7f99101438f0 2023-06-23T18:35:36.619 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.612+0000 7f9915186700 1 -- 172.21.15.155:0/308684334 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0036a0 con 0x7f99101438f0 2023-06-23T18:35:36.621 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.612+0000 7f99183ec700 1 -- 172.21.15.155:0/308684334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99101438f0 msgr2=0x7f99100b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.621 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.612+0000 7f99183ec700 1 --2- 172.21.15.155:0/308684334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99101438f0 0x7f99100b1bb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f990c009840 tx=0x7f990c009b50 comp rx=0 tx=0).stop 2023-06-23T18:35:36.622 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.612+0000 7f99183ec700 1 -- 172.21.15.155:0/308684334 shutdown_connections 2023-06-23T18:35:36.622 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.612+0000 7f99183ec700 1 --2- 172.21.15.155:0/308684334 >> v2:172.21.15.155:3300/0 conn(0x7f99100cedf0 0x7f99100b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.622 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.612+0000 7f99183ec700 1 --2- 172.21.15.155:0/308684334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99101438f0 0x7f99100b1bb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.622 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.612+0000 7f99183ec700 1 -- 172.21.15.155:0/308684334 >> 172.21.15.155:0/308684334 conn(0x7f991006a7b0 msgr2=0x7f9910099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:36.622 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.612+0000 7f99183ec700 1 -- 172.21.15.155:0/308684334 shutdown_connections 2023-06-23T18:35:36.622 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.612+0000 7f99183ec700 1 -- 172.21.15.155:0/308684334 wait complete. 2023-06-23T18:35:36.622 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.613+0000 7f99183ec700 1 Processor -- start 2023-06-23T18:35:36.623 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.614+0000 7f99183ec700 1 -- start start 2023-06-23T18:35:36.623 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.614+0000 7f99183ec700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99100cedf0 0x7f991006f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.623 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.614+0000 7f99183ec700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99100d08c0 0x7f99101f1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.623 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.614+0000 7f99183ec700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99101438f0 0x7f991006caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.623 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.614+0000 7f99183ec700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991006cfe0 con 0x7f99100d08c0 2023-06-23T18:35:36.623 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.614+0000 7f99183ec700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99100c4490 con 0x7f99100cedf0 2023-06-23T18:35:36.623 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.614+0000 7f99183ec700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991006f670 con 0x7f99101438f0 2023-06-23T18:35:36.624 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.614+0000 7f9916188700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99100cedf0 0x7f991006f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.624 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.614+0000 7f9916989700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99101438f0 0x7f991006caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.624 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.614+0000 7f9916188700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99100cedf0 0x7f991006f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40386/0 (socket says 172.21.15.155:40386) 2023-06-23T18:35:36.624 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.614+0000 7f9916188700 1 -- 172.21.15.155:0/150656160 learned_addr learned my addr 172.21.15.155:0/150656160 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-06-23T18:35:36.624 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f9915987700 1 --2- 172.21.15.155:0/150656160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99100d08c0 0x7f99101f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.624 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f9916188700 1 -- 172.21.15.155:0/150656160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99101438f0 msgr2=0x7f991006caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.624 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f9916188700 1 --2- 172.21.15.155:0/150656160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99101438f0 0x7f991006caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.625 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f9916188700 1 -- 172.21.15.155:0/150656160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99100d08c0 msgr2=0x7f99101f1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.625 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f9916188700 1 --2- 172.21.15.155:0/150656160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99100d08c0 0x7f99101f1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.625 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f9916188700 1 -- 172.21.15.155:0/150656160 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9900009500 con 0x7f99100cedf0 2023-06-23T18:35:36.625 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f9916989700 1 --2- 172.21.15.155:0/150656160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99101438f0 0x7f991006caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:36.625 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f9915987700 1 --2- 172.21.15.155:0/150656160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99100d08c0 0x7f99101f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:35:36.625 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f9916188700 1 --2- 172.21.15.155:0/150656160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99100cedf0 0x7f991006f040 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f990000ebb0 tx=0x7f990000eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:36.625 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f9906ffd700 1 -- 172.21.15.155:0/150656160 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9900005a20 con 0x7f99100cedf0 2023-06-23T18:35:36.626 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f9906ffd700 1 -- 172.21.15.155:0/150656160 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9900005b80 con 0x7f99100cedf0 2023-06-23T18:35:36.626 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f99183ec700 1 -- 172.21.15.155:0/150656160 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990c0094a0 con 0x7f99100cedf0 2023-06-23T18:35:36.626 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f9906ffd700 1 -- 172.21.15.155:0/150656160 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990000c850 con 0x7f99100cedf0 2023-06-23T18:35:36.626 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f99183ec700 1 -- 172.21.15.155:0/150656160 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991006fc30 con 0x7f99100cedf0 2023-06-23T18:35:36.626 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f9906ffd700 1 -- 172.21.15.155:0/150656160 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f990000ccb0 con 0x7f99100cedf0 2023-06-23T18:35:36.626 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.615+0000 7f9906ffd700 1 -- 172.21.15.155:0/150656160 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f99000068a0 con 0x7f99100cedf0 2023-06-23T18:35:36.626 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.616+0000 7f99183ec700 1 -- 172.21.15.155:0/150656160 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99100c4600 con 0x7f99100cedf0 2023-06-23T18:35:36.627 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.618+0000 7f9906ffd700 1 -- 172.21.15.155:0/150656160 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f9900020060 con 0x7f99100cedf0 2023-06-23T18:35:36.706 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.701+0000 7f99183ec700 1 -- 172.21.15.155:0/150656160 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d2efa121-6af5-4b4d-b2df-2df7070cc986", "id": 3} v 0) v1 -- 0x7f99100672c0 con 0x7f99100cedf0 2023-06-23T18:35:36.714 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.709+0000 7f9906ffd700 1 -- 172.21.15.155:0/150656160 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d2efa121-6af5-4b4d-b2df-2df7070cc986", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9900006030 con 0x7f99100cedf0 2023-06-23T18:35:36.714 INFO:teuthology.orchestra.run.smithi155.stdout:3 2023-06-23T18:35:36.715 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.710+0000 7f99183ec700 1 -- 172.21.15.155:0/150656160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99100cedf0 msgr2=0x7f991006f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.715 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.710+0000 7f99183ec700 1 --2- 172.21.15.155:0/150656160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99100cedf0 0x7f991006f040 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f990000ebb0 tx=0x7f990000eec0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.716 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.710+0000 7f99183ec700 1 -- 172.21.15.155:0/150656160 shutdown_connections 2023-06-23T18:35:36.716 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.710+0000 7f99183ec700 1 --2- 172.21.15.155:0/150656160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99101438f0 0x7f991006caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.716 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.710+0000 7f99183ec700 1 --2- 172.21.15.155:0/150656160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99100cedf0 0x7f991006f040 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.716 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.710+0000 7f99183ec700 1 --2- 172.21.15.155:0/150656160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99100d08c0 0x7f99101f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.716 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.710+0000 7f99183ec700 1 -- 172.21.15.155:0/150656160 >> 172.21.15.155:0/150656160 conn(0x7f991006a7b0 msgr2=0x7f99100b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:36.716 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.710+0000 7f99183ec700 1 -- 172.21.15.155:0/150656160 shutdown_connections 2023-06-23T18:35:36.716 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.710+0000 7f99183ec700 1 -- 172.21.15.155:0/150656160 wait complete. 2023-06-23T18:35:36.727 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 5ac17afb-4981-4b93-aaf5-8e0224b3d408 4 2023-06-23T18:35:36.767 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:36.762+0000 7f4a314e2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T18:35:36.789 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:36.785+0000 7fb8a0753200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T18:35:36.850 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.845+0000 7f62d17f0700 1 Processor -- start 2023-06-23T18:35:36.850 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.845+0000 7f62d17f0700 1 -- start start 2023-06-23T18:35:36.853 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.846+0000 7f62d17f0700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f62cc1438f0 0x7f62cc0b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.853 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.846+0000 7f62d17f0700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc0b0b10 con 0x7f62cc0d08c0 2023-06-23T18:35:36.854 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.846+0000 7f62d17f0700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc0b0df0 con 0x7f62cc0cedf0 2023-06-23T18:35:36.854 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.846+0000 7f62d17f0700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62cc0b10d0 con 0x7f62cc1438f0 2023-06-23T18:35:36.854 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.846+0000 7f62ca7fc700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f62cc1438f0 0x7f62cc0b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.855 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.846+0000 7f62ca7fc700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f62cc1438f0 0x7f62cc0b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40394/0 (socket says 172.21.15.155:40394) 2023-06-23T18:35:36.855 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.846+0000 7f62ca7fc700 1 -- 172.21.15.155:0/4097622845 learned_addr learned my addr 172.21.15.155:0/4097622845 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-06-23T18:35:36.855 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.846+0000 7f62caffd700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f62cc0cedf0 0x7f62cc0b02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:57124/0 (socket says 172.21.15.155:57124) 2023-06-23T18:35:36.855 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.847+0000 7f62ca7fc700 1 -- 172.21.15.155:0/4097622845 >> v1:172.21.15.155:6789/0 conn(0x7f62cc0cedf0 legacy=0x7f62cc0b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.856 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.847+0000 7f62ca7fc700 1 -- 172.21.15.155:0/4097622845 >> v1:172.21.15.79:6789/0 conn(0x7f62cc0d08c0 legacy=0x7f62cc0d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.856 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.847+0000 7f62ca7fc700 1 -- 172.21.15.155:0/4097622845 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc0b13b0 con 0x7f62cc1438f0 2023-06-23T18:35:36.856 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.847+0000 7f62ca7fc700 1 --2- 172.21.15.155:0/4097622845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc1438f0 0x7f62cc0b1bb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f62c00098f0 tx=0x7f62c0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23dc4bf4188e029 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:36.856 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.847+0000 7f62c9ffb700 1 -- 172.21.15.155:0/4097622845 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c000b8a0 con 0x7f62cc1438f0 2023-06-23T18:35:36.857 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.847+0000 7f62c9ffb700 1 -- 172.21.15.155:0/4097622845 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c000ba00 con 0x7f62cc1438f0 2023-06-23T18:35:36.857 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.847+0000 7f62c9ffb700 1 -- 172.21.15.155:0/4097622845 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c000bd90 con 0x7f62cc1438f0 2023-06-23T18:35:36.857 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.848+0000 7f62d17f0700 1 -- 172.21.15.155:0/4097622845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc1438f0 msgr2=0x7f62cc0b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.858 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.848+0000 7f62d17f0700 1 --2- 172.21.15.155:0/4097622845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc1438f0 0x7f62cc0b1bb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f62c00098f0 tx=0x7f62c0009c00 comp rx=0 tx=0).stop 2023-06-23T18:35:36.858 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.848+0000 7f62d17f0700 1 -- 172.21.15.155:0/4097622845 shutdown_connections 2023-06-23T18:35:36.858 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.848+0000 7f62d17f0700 1 --2- 172.21.15.155:0/4097622845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc1438f0 0x7f62cc0b1bb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.858 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.848+0000 7f62d17f0700 1 -- 172.21.15.155:0/4097622845 >> 172.21.15.155:0/4097622845 conn(0x7f62cc06a7b0 msgr2=0x7f62cc099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:36.859 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.848+0000 7f62d17f0700 1 -- 172.21.15.155:0/4097622845 shutdown_connections 2023-06-23T18:35:36.859 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.848+0000 7f62d17f0700 1 -- 172.21.15.155:0/4097622845 wait complete. 2023-06-23T18:35:36.862 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.849+0000 7f62d17f0700 1 Processor -- start 2023-06-23T18:35:36.863 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.850+0000 7f62d17f0700 1 -- start start 2023-06-23T18:35:36.863 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.850+0000 7f62d17f0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62cc0cedf0 0x7f62cc154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.863 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.850+0000 7f62d17f0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc0d08c0 0x7f62cc1f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.863 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.850+0000 7f62d17f0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62cc1438f0 0x7f62cc15c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:36.863 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.850+0000 7f62d17f0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc15b4f0 con 0x7f62cc0cedf0 2023-06-23T18:35:36.863 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.850+0000 7f62d17f0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc15b9f0 con 0x7f62cc0d08c0 2023-06-23T18:35:36.864 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.850+0000 7f62d17f0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0ae350 con 0x7f62cc1438f0 2023-06-23T18:35:36.864 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62ca7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc0d08c0 0x7f62cc1f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.864 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62ca7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc0d08c0 0x7f62cc1f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40400/0 (socket says 172.21.15.155:40400) 2023-06-23T18:35:36.864 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62ca7fc700 1 -- 172.21.15.155:0/1638825807 learned_addr learned my addr 172.21.15.155:0/1638825807 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-06-23T18:35:36.864 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62cb7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62cc1438f0 0x7f62cc15c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.864 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62caffd700 1 --2- 172.21.15.155:0/1638825807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62cc0cedf0 0x7f62cc154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:36.864 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62cb7fe700 1 -- 172.21.15.155:0/1638825807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc0d08c0 msgr2=0x7f62cc1f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.865 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62cb7fe700 1 --2- 172.21.15.155:0/1638825807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc0d08c0 0x7f62cc1f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.865 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62cb7fe700 1 -- 172.21.15.155:0/1638825807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62cc0cedf0 msgr2=0x7f62cc154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.865 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62cb7fe700 1 --2- 172.21.15.155:0/1638825807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62cc0cedf0 0x7f62cc154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.865 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62cb7fe700 1 -- 172.21.15.155:0/1638825807 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c0009550 con 0x7f62cc1438f0 2023-06-23T18:35:36.865 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62ca7fc700 1 --2- 172.21.15.155:0/1638825807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc0d08c0 0x7f62cc1f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:36.865 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62cb7fe700 1 --2- 172.21.15.155:0/1638825807 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62cc1438f0 0x7f62cc15c9e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f62c400cb90 tx=0x7f62c400cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:36.865 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62b37fe700 1 -- 172.21.15.155:0/1638825807 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4003c00 con 0x7f62cc1438f0 2023-06-23T18:35:36.865 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62b37fe700 1 -- 172.21.15.155:0/1638825807 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c4003d60 con 0x7f62cc1438f0 2023-06-23T18:35:36.866 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62d17f0700 1 -- 172.21.15.155:0/1638825807 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62cc1f5450 con 0x7f62cc1438f0 2023-06-23T18:35:36.866 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62b37fe700 1 -- 172.21.15.155:0/1638825807 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c400aa80 con 0x7f62cc1438f0 2023-06-23T18:35:36.866 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62d17f0700 1 -- 172.21.15.155:0/1638825807 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62cc15c1e0 con 0x7f62cc1438f0 2023-06-23T18:35:36.866 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.851+0000 7f62b37fe700 1 -- 172.21.15.155:0/1638825807 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f62c40050d0 con 0x7f62cc1438f0 2023-06-23T18:35:36.866 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.852+0000 7f62b37fe700 1 -- 172.21.15.155:0/1638825807 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f62c4012a20 con 0x7f62cc1438f0 2023-06-23T18:35:36.866 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.852+0000 7f62d17f0700 1 -- 172.21.15.155:0/1638825807 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62cc0672c0 con 0x7f62cc1438f0 2023-06-23T18:35:36.866 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.854+0000 7f62b37fe700 1 -- 172.21.15.155:0/1638825807 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f62c4012c80 con 0x7f62cc1438f0 2023-06-23T18:35:36.943 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.939+0000 7f62d17f0700 1 -- 172.21.15.155:0/1638825807 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5ac17afb-4981-4b93-aaf5-8e0224b3d408", "id": 4} v 0) v1 -- 0x7f62cc09a010 con 0x7f62cc1438f0 2023-06-23T18:35:36.950 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.946+0000 7f62b37fe700 1 -- 172.21.15.155:0/1638825807 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5ac17afb-4981-4b93-aaf5-8e0224b3d408", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f62c4012e20 con 0x7f62cc1438f0 2023-06-23T18:35:36.951 INFO:teuthology.orchestra.run.smithi155.stdout:4 2023-06-23T18:35:36.952 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.947+0000 7f62d17f0700 1 -- 172.21.15.155:0/1638825807 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62cc1438f0 msgr2=0x7f62cc15c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:36.952 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.947+0000 7f62d17f0700 1 --2- 172.21.15.155:0/1638825807 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62cc1438f0 0x7f62cc15c9e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f62c400cb90 tx=0x7f62c400cea0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.952 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.947+0000 7f62d17f0700 1 -- 172.21.15.155:0/1638825807 shutdown_connections 2023-06-23T18:35:36.953 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.947+0000 7f62d17f0700 1 --2- 172.21.15.155:0/1638825807 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62cc1438f0 0x7f62cc15c9e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.953 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.947+0000 7f62d17f0700 1 --2- 172.21.15.155:0/1638825807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62cc0cedf0 0x7f62cc154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.953 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.947+0000 7f62d17f0700 1 --2- 172.21.15.155:0/1638825807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc0d08c0 0x7f62cc1f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:36.953 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.947+0000 7f62d17f0700 1 -- 172.21.15.155:0/1638825807 >> 172.21.15.155:0/1638825807 conn(0x7f62cc06a7b0 msgr2=0x7f62cc0b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:36.953 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.947+0000 7f62d17f0700 1 -- 172.21.15.155:0/1638825807 shutdown_connections 2023-06-23T18:35:36.954 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:36.947+0000 7f62d17f0700 1 -- 172.21.15.155:0/1638825807 wait complete. 2023-06-23T18:35:36.964 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new e602dbcd-be18-4aa0-a3f6-8bff41056208 5 2023-06-23T18:35:37.084 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.079+0000 7f2acfe62700 1 Processor -- start 2023-06-23T18:35:37.084 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.079+0000 7f2acfe62700 1 -- start start 2023-06-23T18:35:37.085 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.080+0000 7f2acfe62700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2ac80cb890 0x7f2ac80cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.086 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.080+0000 7f2acfe62700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac80cc9f0 con 0x7f2ac80cc060 2023-06-23T18:35:37.086 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.080+0000 7f2acfe62700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac815a560 con 0x7f2ac8156ae0 2023-06-23T18:35:37.086 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.080+0000 7f2acfe62700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac80cbcd0 con 0x7f2ac80cb890 2023-06-23T18:35:37.086 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.080+0000 7f2ace3ff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2ac80cb890 0x7f2ac80cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.086 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.080+0000 7f2acdbfe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f2ac8156ae0 0x7f2ac815d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:57136/0 (socket says 172.21.15.155:57136) 2023-06-23T18:35:37.086 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.080+0000 7f2acdbfe700 1 -- 172.21.15.155:0/299196930 learned_addr learned my addr 172.21.15.155:0/299196930 (peer_addr_for_me v1:172.21.15.155:0/0) 2023-06-23T18:35:37.087 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.081+0000 7f2ace3ff700 1 -- 172.21.15.155:0/299196930 >> v1:172.21.15.155:6789/0 conn(0x7f2ac8156ae0 legacy=0x7f2ac815d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.087 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.081+0000 7f2ace3ff700 1 -- 172.21.15.155:0/299196930 >> v1:172.21.15.79:6789/0 conn(0x7f2ac80cc060 legacy=0x7f2ac8071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.087 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.081+0000 7f2ace3ff700 1 -- 172.21.15.155:0/299196930 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac8159950 con 0x7f2ac80cb890 2023-06-23T18:35:37.087 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.081+0000 7f2accbfc700 1 -- 172.21.15.155:0/299196930 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500922752 0 0) 0x7f2ac815a560 con 0x7f2ac8156ae0 2023-06-23T18:35:37.087 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.081+0000 7f2ace3ff700 1 --2- 172.21.15.155:0/299196930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac80cb890 0x7f2ac80cc4b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009d90 tx=0x7f2ac000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b96441406461a980 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:37.087 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.081+0000 7f2accbfc700 1 -- 172.21.15.155:0/299196930 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac000e070 con 0x7f2ac80cb890 2023-06-23T18:35:37.087 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.081+0000 7f2accbfc700 1 -- 172.21.15.155:0/299196930 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac000bbd0 con 0x7f2ac80cb890 2023-06-23T18:35:37.087 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.081+0000 7f2accbfc700 1 -- 172.21.15.155:0/299196930 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0012430 con 0x7f2ac80cb890 2023-06-23T18:35:37.088 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.082+0000 7f2acfe62700 1 -- 172.21.15.155:0/299196930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac80cb890 msgr2=0x7f2ac80cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.082+0000 7f2acfe62700 1 --2- 172.21.15.155:0/299196930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac80cb890 0x7f2ac80cc4b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009d90 tx=0x7f2ac000b330 comp rx=0 tx=0).stop 2023-06-23T18:35:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.082+0000 7f2acfe62700 1 -- 172.21.15.155:0/299196930 shutdown_connections 2023-06-23T18:35:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.082+0000 7f2acfe62700 1 --2- 172.21.15.155:0/299196930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac80cb890 0x7f2ac80cc4b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.082+0000 7f2acfe62700 1 -- 172.21.15.155:0/299196930 >> 172.21.15.155:0/299196930 conn(0x7f2ac806afa0 msgr2=0x7f2ac806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.082+0000 7f2acfe62700 1 -- 172.21.15.155:0/299196930 shutdown_connections 2023-06-23T18:35:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.082+0000 7f2acfe62700 1 -- 172.21.15.155:0/299196930 wait complete. 2023-06-23T18:35:37.089 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.084+0000 7f2acfe62700 1 Processor -- start 2023-06-23T18:35:37.090 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.084+0000 7f2acfe62700 1 -- start start 2023-06-23T18:35:37.091 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acfe62700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ac80cb890 0x7f2ac80ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.092 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acfe62700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac80cc060 0x7f2ac81477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.092 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acfe62700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ac8156ae0 0x7f2ac8149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.092 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acfe62700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac8147550 con 0x7f2ac8156ae0 2023-06-23T18:35:37.092 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acfe62700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac814a2e0 con 0x7f2ac80cc060 2023-06-23T18:35:37.092 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acfe62700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac814a5c0 con 0x7f2ac80cb890 2023-06-23T18:35:37.092 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acd3fd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac80cc060 0x7f2ac81477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.092 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acdbfe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ac80cb890 0x7f2ac80ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acd3fd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac80cc060 0x7f2ac81477d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40412/0 (socket says 172.21.15.155:40412) 2023-06-23T18:35:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acd3fd700 1 -- 172.21.15.155:0/2392168576 learned_addr learned my addr 172.21.15.155:0/2392168576 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-06-23T18:35:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2ace3ff700 1 --2- 172.21.15.155:0/2392168576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ac8156ae0 0x7f2ac8149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acd3fd700 1 -- 172.21.15.155:0/2392168576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ac80cb890 msgr2=0x7f2ac80ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acd3fd700 1 --2- 172.21.15.155:0/2392168576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ac80cb890 0x7f2ac80ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acd3fd700 1 -- 172.21.15.155:0/2392168576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ac8156ae0 msgr2=0x7f2ac8149860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acd3fd700 1 --2- 172.21.15.155:0/2392168576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ac8156ae0 0x7f2ac8149860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.093 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acd3fd700 1 -- 172.21.15.155:0/2392168576 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac0009a70 con 0x7f2ac80cc060 2023-06-23T18:35:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.085+0000 7f2acdbfe700 1 --2- 172.21.15.155:0/2392168576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ac80cb890 0x7f2ac80ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.086+0000 7f2acd3fd700 1 --2- 172.21.15.155:0/2392168576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac80cc060 0x7f2ac81477d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800c4e0 tx=0x7f2ab800c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.086+0000 7f2abe7fc700 1 -- 172.21.15.155:0/2392168576 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab80052d0 con 0x7f2ac80cc060 2023-06-23T18:35:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.086+0000 7f2abe7fc700 1 -- 172.21.15.155:0/2392168576 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab8005430 con 0x7f2ac80cc060 2023-06-23T18:35:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.086+0000 7f2abe7fc700 1 -- 172.21.15.155:0/2392168576 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab80038c0 con 0x7f2ac80cc060 2023-06-23T18:35:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.086+0000 7f2acfe62700 1 -- 172.21.15.155:0/2392168576 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac80b4070 con 0x7f2ac80cc060 2023-06-23T18:35:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.086+0000 7f2acfe62700 1 -- 172.21.15.155:0/2392168576 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac80b3d40 con 0x7f2ac80cc060 2023-06-23T18:35:37.094 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.086+0000 7f2abe7fc700 1 -- 172.21.15.155:0/2392168576 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2ab8003d20 con 0x7f2ac80cc060 2023-06-23T18:35:37.095 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.086+0000 7f2abe7fc700 1 -- 172.21.15.155:0/2392168576 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f2ab8011d60 con 0x7f2ac80cc060 2023-06-23T18:35:37.095 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.087+0000 7f2acfe62700 1 -- 172.21.15.155:0/2392168576 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac80672c0 con 0x7f2ac80cc060 2023-06-23T18:35:37.095 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.089+0000 7f2abe7fc700 1 -- 172.21.15.155:0/2392168576 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f2ab8017070 con 0x7f2ac80cc060 2023-06-23T18:35:37.180 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.176+0000 7f2acfe62700 1 -- 172.21.15.155:0/2392168576 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e602dbcd-be18-4aa0-a3f6-8bff41056208", "id": 5} v 0) v1 -- 0x7f2ac80b3ed0 con 0x7f2ac80cc060 2023-06-23T18:35:37.199 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.194+0000 7f2abe7fc700 1 -- 172.21.15.155:0/2392168576 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e602dbcd-be18-4aa0-a3f6-8bff41056208", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2ac80b3ed0 con 0x7f2ac80cc060 2023-06-23T18:35:37.200 INFO:teuthology.orchestra.run.smithi155.stdout:5 2023-06-23T18:35:37.200 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.195+0000 7f2acfe62700 1 -- 172.21.15.155:0/2392168576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac80cc060 msgr2=0x7f2ac81477d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.201 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.195+0000 7f2acfe62700 1 --2- 172.21.15.155:0/2392168576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac80cc060 0x7f2ac81477d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800c4e0 tx=0x7f2ab800c7f0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.201 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.195+0000 7f2acfe62700 1 -- 172.21.15.155:0/2392168576 shutdown_connections 2023-06-23T18:35:37.201 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.195+0000 7f2acfe62700 1 --2- 172.21.15.155:0/2392168576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ac8156ae0 0x7f2ac8149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.201 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.195+0000 7f2acfe62700 1 --2- 172.21.15.155:0/2392168576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ac80cb890 0x7f2ac80ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.201 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.195+0000 7f2acfe62700 1 --2- 172.21.15.155:0/2392168576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac80cc060 0x7f2ac81477d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.201 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.195+0000 7f2acfe62700 1 -- 172.21.15.155:0/2392168576 >> 172.21.15.155:0/2392168576 conn(0x7f2ac806afa0 msgr2=0x7f2ac80cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:37.202 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.196+0000 7f2acfe62700 1 -- 172.21.15.155:0/2392168576 shutdown_connections 2023-06-23T18:35:37.202 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.196+0000 7f2acfe62700 1 -- 172.21.15.155:0/2392168576 wait complete. 2023-06-23T18:35:37.215 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new dfcc906c-8979-4f61-b01a-88540da6a253 6 2023-06-23T18:35:37.319 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:37.314+0000 7f4a314e2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T18:35:37.349 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.345+0000 7ff1ccff6700 1 Processor -- start 2023-06-23T18:35:37.350 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.345+0000 7ff1ccff6700 1 -- start start 2023-06-23T18:35:37.356 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1ccff6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1c81569c0 0x7ff1c81608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.357 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1ccff6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff1c80cd440 0x7ff1c8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.358 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1ccff6700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c80cdc90 con 0x7ff1c815d580 2023-06-23T18:35:37.358 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1ccff6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c80cf570 con 0x7ff1c80cd440 2023-06-23T18:35:37.358 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1ccff6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c815c8b0 con 0x7ff1c81569c0 2023-06-23T18:35:37.359 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1c6d9d700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7ff1c815d580 0x7ff1c80b12f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:57152/0 (socket says 172.21.15.155:57152) 2023-06-23T18:35:37.359 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1c6d9d700 1 -- 172.21.15.155:0/447183393 learned_addr learned my addr 172.21.15.155:0/447183393 (peer_addr_for_me v1:172.21.15.155:0/0) 2023-06-23T18:35:37.359 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1c659c700 1 --2- 172.21.15.155:0/447183393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1c81569c0 0x7ff1c81608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.359 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1c5d9b700 1 --2- 172.21.15.155:0/447183393 >> v2:172.21.15.79:3300/0 conn(0x7ff1c80cd440 0x7ff1c8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.360 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1c559a700 1 -- 172.21.15.155:0/447183393 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307801633 0 0) 0x7ff1c0001490 con 0x7ff1c815d580 2023-06-23T18:35:37.360 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1c559a700 1 -- 172.21.15.155:0/447183393 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1b00035b0 con 0x7ff1c815d580 2023-06-23T18:35:37.361 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1c659c700 1 -- 172.21.15.155:0/447183393 >> v2:172.21.15.79:3300/0 conn(0x7ff1c80cd440 msgr2=0x7ff1c8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:37.361 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1c659c700 1 --2- 172.21.15.155:0/447183393 >> v2:172.21.15.79:3300/0 conn(0x7ff1c80cd440 0x7ff1c8069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.361 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1c659c700 1 -- 172.21.15.155:0/447183393 >> v1:172.21.15.155:6789/0 conn(0x7ff1c815d580 legacy=0x7ff1c80b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.361 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1c659c700 1 -- 172.21.15.155:0/447183393 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c815d0f0 con 0x7ff1c81569c0 2023-06-23T18:35:37.362 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1c659c700 1 --2- 172.21.15.155:0/447183393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1c81569c0 0x7ff1c81608b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009840 tx=0x7ff1b8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c06339f3e359b0b7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:37.362 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.346+0000 7ff1c559a700 1 -- 172.21.15.155:0/447183393 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8011830 con 0x7ff1c81569c0 2023-06-23T18:35:37.362 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.347+0000 7ff1c559a700 1 -- 172.21.15.155:0/447183393 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b8011990 con 0x7ff1c81569c0 2023-06-23T18:35:37.363 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.347+0000 7ff1c559a700 1 -- 172.21.15.155:0/447183393 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8011d20 con 0x7ff1c81569c0 2023-06-23T18:35:37.363 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.347+0000 7ff1ccff6700 1 -- 172.21.15.155:0/447183393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1c81569c0 msgr2=0x7ff1c81608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.363 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.347+0000 7ff1ccff6700 1 --2- 172.21.15.155:0/447183393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1c81569c0 0x7ff1c81608b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009840 tx=0x7ff1b8009b50 comp rx=0 tx=0).stop 2023-06-23T18:35:37.363 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.347+0000 7ff1ccff6700 1 -- 172.21.15.155:0/447183393 shutdown_connections 2023-06-23T18:35:37.364 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.347+0000 7ff1ccff6700 1 --2- 172.21.15.155:0/447183393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1c81569c0 0x7ff1c81608b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.364 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.347+0000 7ff1ccff6700 1 --2- 172.21.15.155:0/447183393 >> v2:172.21.15.79:3300/0 conn(0x7ff1c80cd440 0x7ff1c8069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.364 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.347+0000 7ff1ccff6700 1 -- 172.21.15.155:0/447183393 >> 172.21.15.155:0/447183393 conn(0x7ff1c806afa0 msgr2=0x7ff1c806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:37.364 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.347+0000 7ff1ccff6700 1 -- 172.21.15.155:0/447183393 shutdown_connections 2023-06-23T18:35:37.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.347+0000 7ff1ccff6700 1 -- 172.21.15.155:0/447183393 wait complete. 2023-06-23T18:35:37.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.349+0000 7ff1ccff6700 1 Processor -- start 2023-06-23T18:35:37.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.349+0000 7ff1ccff6700 1 -- start start 2023-06-23T18:35:37.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.349+0000 7ff1ccff6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1c80cd440 0x7ff1c814ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.349+0000 7ff1ccff6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1c81569c0 0x7ff1c80c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.349+0000 7ff1ccff6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1c815d580 0x7ff1c80c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.349+0000 7ff1ccff6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80c7ab0 con 0x7ff1c80cd440 2023-06-23T18:35:37.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.349+0000 7ff1ccff6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80c4500 con 0x7ff1c815d580 2023-06-23T18:35:37.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.349+0000 7ff1ccff6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80c62c0 con 0x7ff1c81569c0 2023-06-23T18:35:37.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1c5d9b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1c81569c0 0x7ff1c80c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1c5d9b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1c81569c0 0x7ff1c80c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:56214/0 (socket says 172.21.15.155:56214) 2023-06-23T18:35:37.368 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1c6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1c815d580 0x7ff1c80c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.368 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1c5d9b700 1 -- 172.21.15.155:0/364124443 learned_addr learned my addr 172.21.15.155:0/364124443 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-06-23T18:35:37.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1c659c700 1 --2- 172.21.15.155:0/364124443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1c80cd440 0x7ff1c814ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1c5d9b700 1 -- 172.21.15.155:0/364124443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1c815d580 msgr2=0x7ff1c80c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1c5d9b700 1 --2- 172.21.15.155:0/364124443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1c815d580 0x7ff1c80c3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1c5d9b700 1 -- 172.21.15.155:0/364124443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1c80cd440 msgr2=0x7ff1c814ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1c5d9b700 1 --2- 172.21.15.155:0/364124443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1c80cd440 0x7ff1c814ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1c5d9b700 1 -- 172.21.15.155:0/364124443 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b80094a0 con 0x7ff1c81569c0 2023-06-23T18:35:37.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1c6d9d700 1 --2- 172.21.15.155:0/364124443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1c815d580 0x7ff1c80c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:37.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1c5d9b700 1 --2- 172.21.15.155:0/364124443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1c81569c0 0x7ff1c80c7570 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00e9e0 tx=0x7ff1bc00ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:37.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1b6ffd700 1 -- 172.21.15.155:0/364124443 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc009710 con 0x7ff1c81569c0 2023-06-23T18:35:37.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1b6ffd700 1 -- 172.21.15.155:0/364124443 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc00ee80 con 0x7ff1c81569c0 2023-06-23T18:35:37.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1b6ffd700 1 -- 172.21.15.155:0/364124443 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc00ca50 con 0x7ff1c81569c0 2023-06-23T18:35:37.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1ccff6700 1 -- 172.21.15.155:0/364124443 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c80c5500 con 0x7ff1c81569c0 2023-06-23T18:35:37.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.350+0000 7ff1ccff6700 1 -- 172.21.15.155:0/364124443 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c80c5ac0 con 0x7ff1c81569c0 2023-06-23T18:35:37.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.352+0000 7ff1b6ffd700 1 -- 172.21.15.155:0/364124443 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff1bc013470 con 0x7ff1c81569c0 2023-06-23T18:35:37.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.352+0000 7ff1b6ffd700 1 -- 172.21.15.155:0/364124443 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7ff1bc01cc00 con 0x7ff1c81569c0 2023-06-23T18:35:37.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.352+0000 7ff1ccff6700 1 -- 172.21.15.155:0/364124443 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c80aeda0 con 0x7ff1c81569c0 2023-06-23T18:35:37.372 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.354+0000 7ff1b6ffd700 1 -- 172.21.15.155:0/364124443 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ff1bc021060 con 0x7ff1c81569c0 2023-06-23T18:35:37.386 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:37.382+0000 7fb8a0753200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T18:35:37.439 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.435+0000 7ff1ccff6700 1 -- 172.21.15.155:0/364124443 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dfcc906c-8979-4f61-b01a-88540da6a253", "id": 6} v 0) v1 -- 0x7ff1c80c6690 con 0x7ff1c81569c0 2023-06-23T18:35:37.447 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.442+0000 7ff1b6ffd700 1 -- 172.21.15.155:0/364124443 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dfcc906c-8979-4f61-b01a-88540da6a253", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff1bc02b0b0 con 0x7ff1c81569c0 2023-06-23T18:35:37.447 INFO:teuthology.orchestra.run.smithi155.stdout:6 2023-06-23T18:35:37.448 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.444+0000 7ff1ccff6700 1 -- 172.21.15.155:0/364124443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1c81569c0 msgr2=0x7ff1c80c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.449 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.444+0000 7ff1ccff6700 1 --2- 172.21.15.155:0/364124443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1c81569c0 0x7ff1c80c7570 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00e9e0 tx=0x7ff1bc00ecf0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.449 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.444+0000 7ff1ccff6700 1 -- 172.21.15.155:0/364124443 shutdown_connections 2023-06-23T18:35:37.449 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.444+0000 7ff1ccff6700 1 --2- 172.21.15.155:0/364124443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1c815d580 0x7ff1c80c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.449 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.444+0000 7ff1ccff6700 1 --2- 172.21.15.155:0/364124443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1c80cd440 0x7ff1c814ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.449 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.444+0000 7ff1ccff6700 1 --2- 172.21.15.155:0/364124443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1c81569c0 0x7ff1c80c7570 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.449 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.444+0000 7ff1ccff6700 1 -- 172.21.15.155:0/364124443 >> 172.21.15.155:0/364124443 conn(0x7ff1c806afa0 msgr2=0x7ff1c815ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:37.449 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.444+0000 7ff1ccff6700 1 -- 172.21.15.155:0/364124443 shutdown_connections 2023-06-23T18:35:37.450 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.444+0000 7ff1ccff6700 1 -- 172.21.15.155:0/364124443 wait complete. 2023-06-23T18:35:37.461 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 5858f908-e534-4ab8-9b82-ff6b01ff58b2 7 2023-06-23T18:35:37.473 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:37.468+0000 7f4a314e2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T18:35:37.550 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:37.546+0000 7fb8a0753200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T18:35:37.586 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.581+0000 7f36912c0700 1 Processor -- start 2023-06-23T18:35:37.587 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.582+0000 7f36912c0700 1 -- start start 2023-06-23T18:35:37.589 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.582+0000 7f36912c0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f368c156ae0 0x7f368c15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.589 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.582+0000 7f36912c0700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f368c0cc4d0 0x7f368c071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.589 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.582+0000 7f36912c0700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368c15daf0 con 0x7f368c0cc060 2023-06-23T18:35:37.590 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.582+0000 7f36912c0700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368c15a560 con 0x7f368c0cc4d0 2023-06-23T18:35:37.590 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.582+0000 7f36912c0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c159950 con 0x7f368c156ae0 2023-06-23T18:35:37.590 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.582+0000 7f368affd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f368c156ae0 0x7f368c15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.590 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.583+0000 7f368affd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f368c156ae0 0x7f368c15d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:56218/0 (socket says 172.21.15.155:56218) 2023-06-23T18:35:37.590 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.583+0000 7f368b7fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f368c0cc4d0 0x7f368c071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.591 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.583+0000 7f368affd700 1 -- 172.21.15.155:0/2630562334 learned_addr learned my addr 172.21.15.155:0/2630562334 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-06-23T18:35:37.591 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.583+0000 7f368affd700 1 -- 172.21.15.155:0/2630562334 >> v2:172.21.15.155:3300/0 conn(0x7f368c0cc4d0 msgr2=0x7f368c071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:37.591 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.583+0000 7f368affd700 1 --2- 172.21.15.155:0/2630562334 >> v2:172.21.15.155:3300/0 conn(0x7f368c0cc4d0 0x7f368c071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.591 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.583+0000 7f368affd700 1 -- 172.21.15.155:0/2630562334 >> v1:172.21.15.155:6789/0 conn(0x7f368c0cc060 legacy=0x7f368c071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.591 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.583+0000 7f368affd700 1 -- 172.21.15.155:0/2630562334 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c159c30 con 0x7f368c156ae0 2023-06-23T18:35:37.591 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.583+0000 7f368b7fe700 1 --2- 172.21.15.155:0/2630562334 >> v2:172.21.15.155:3300/0 conn(0x7f368c0cc4d0 0x7f368c071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:37.591 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.583+0000 7f368affd700 1 --2- 172.21.15.155:0/2630562334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f368c156ae0 0x7f368c15d5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f367c0098f0 tx=0x7f367c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=957aab097afdb21e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:37.591 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.583+0000 7f3689ffb700 1 -- 172.21.15.155:0/2630562334 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c011830 con 0x7f368c156ae0 2023-06-23T18:35:37.592 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.583+0000 7f3689ffb700 1 -- 172.21.15.155:0/2630562334 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c011990 con 0x7f368c156ae0 2023-06-23T18:35:37.592 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.583+0000 7f3689ffb700 1 -- 172.21.15.155:0/2630562334 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c011d20 con 0x7f368c156ae0 2023-06-23T18:35:37.592 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.584+0000 7f36912c0700 1 -- 172.21.15.155:0/2630562334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f368c156ae0 msgr2=0x7f368c15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.592 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.584+0000 7f36912c0700 1 --2- 172.21.15.155:0/2630562334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f368c156ae0 0x7f368c15d5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f367c0098f0 tx=0x7f367c009c00 comp rx=0 tx=0).stop 2023-06-23T18:35:37.592 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.584+0000 7f36912c0700 1 -- 172.21.15.155:0/2630562334 shutdown_connections 2023-06-23T18:35:37.593 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.584+0000 7f36912c0700 1 --2- 172.21.15.155:0/2630562334 >> v2:172.21.15.155:3300/0 conn(0x7f368c0cc4d0 0x7f368c071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.593 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.584+0000 7f36912c0700 1 --2- 172.21.15.155:0/2630562334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f368c156ae0 0x7f368c15d5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.593 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.584+0000 7f36912c0700 1 -- 172.21.15.155:0/2630562334 >> 172.21.15.155:0/2630562334 conn(0x7f368c06afa0 msgr2=0x7f368c06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:37.593 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.584+0000 7f36912c0700 1 -- 172.21.15.155:0/2630562334 shutdown_connections 2023-06-23T18:35:37.593 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.584+0000 7f36912c0700 1 -- 172.21.15.155:0/2630562334 wait complete. 2023-06-23T18:35:37.596 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.585+0000 7f36912c0700 1 Processor -- start 2023-06-23T18:35:37.597 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.585+0000 7f36912c0700 1 -- start start 2023-06-23T18:35:37.597 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.586+0000 7f36912c0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f368c0cc060 0x7f368c0b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.597 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.586+0000 7f36912c0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f368c0cc4d0 0x7f368c154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.598 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.586+0000 7f36912c0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f368c156ae0 0x7f368c1e44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.598 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.586+0000 7f36912c0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0b3e90 con 0x7f368c0cc060 2023-06-23T18:35:37.598 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.586+0000 7f36912c0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0b3250 con 0x7f368c156ae0 2023-06-23T18:35:37.598 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.586+0000 7f36912c0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c1554c0 con 0x7f368c0cc4d0 2023-06-23T18:35:37.599 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.586+0000 7f368a7fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f368c0cc4d0 0x7f368c154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.599 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.586+0000 7f368b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f368c156ae0 0x7f368c1e44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.599 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.586+0000 7f368a7fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f368c0cc4d0 0x7f368c154f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:56226/0 (socket says 172.21.15.155:56226) 2023-06-23T18:35:37.600 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.586+0000 7f368a7fc700 1 -- 172.21.15.155:0/3488651760 learned_addr learned my addr 172.21.15.155:0/3488651760 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-06-23T18:35:37.600 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.586+0000 7f368b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f368c156ae0 0x7f368c1e44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:40438/0 (socket says 172.21.15.155:40438) 2023-06-23T18:35:37.600 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.586+0000 7f368affd700 1 --2- 172.21.15.155:0/3488651760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f368c0cc060 0x7f368c0b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.601 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.587+0000 7f368a7fc700 1 -- 172.21.15.155:0/3488651760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f368c156ae0 msgr2=0x7f368c1e44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.601 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.587+0000 7f368a7fc700 1 --2- 172.21.15.155:0/3488651760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f368c156ae0 0x7f368c1e44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.601 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.587+0000 7f368a7fc700 1 -- 172.21.15.155:0/3488651760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f368c0cc060 msgr2=0x7f368c0b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.602 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.587+0000 7f368a7fc700 1 --2- 172.21.15.155:0/3488651760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f368c0cc060 0x7f368c0b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.602 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.587+0000 7f368a7fc700 1 -- 172.21.15.155:0/3488651760 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3684009a70 con 0x7f368c0cc4d0 2023-06-23T18:35:37.602 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.587+0000 7f368b7fe700 1 --2- 172.21.15.155:0/3488651760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f368c156ae0 0x7f368c1e44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:37.603 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.587+0000 7f368a7fc700 1 --2- 172.21.15.155:0/3488651760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f368c0cc4d0 0x7f368c154f80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f368000c4e0 tx=0x7f368000c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:37.603 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.587+0000 7f36737fe700 1 -- 172.21.15.155:0/3488651760 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36800052d0 con 0x7f368c0cc4d0 2023-06-23T18:35:37.603 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.587+0000 7f36737fe700 1 -- 172.21.15.155:0/3488651760 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3680005430 con 0x7f368c0cc4d0 2023-06-23T18:35:37.603 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.587+0000 7f36912c0700 1 -- 172.21.15.155:0/3488651760 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367c009550 con 0x7f368c0cc4d0 2023-06-23T18:35:37.603 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.587+0000 7f36737fe700 1 -- 172.21.15.155:0/3488651760 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36800038c0 con 0x7f368c0cc4d0 2023-06-23T18:35:37.603 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.587+0000 7f36912c0700 1 -- 172.21.15.155:0/3488651760 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c0b37d0 con 0x7f368c0cc4d0 2023-06-23T18:35:37.604 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.587+0000 7f36737fe700 1 -- 172.21.15.155:0/3488651760 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3680003d20 con 0x7f368c0cc4d0 2023-06-23T18:35:37.604 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.587+0000 7f36737fe700 1 -- 172.21.15.155:0/3488651760 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f3680011d60 con 0x7f368c0cc4d0 2023-06-23T18:35:37.604 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.588+0000 7f36912c0700 1 -- 172.21.15.155:0/3488651760 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c0672c0 con 0x7f368c0cc4d0 2023-06-23T18:35:37.604 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.590+0000 7f36737fe700 1 -- 172.21.15.155:0/3488651760 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f3680017070 con 0x7f368c0cc4d0 2023-06-23T18:35:37.678 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.674+0000 7f36912c0700 1 -- 172.21.15.155:0/3488651760 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5858f908-e534-4ab8-9b82-ff6b01ff58b2", "id": 7} v 0) v1 -- 0x7f368c0b1be0 con 0x7f368c0cc4d0 2023-06-23T18:35:37.686 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.682+0000 7f36737fe700 1 -- 172.21.15.155:0/3488651760 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5858f908-e534-4ab8-9b82-ff6b01ff58b2", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3680011590 con 0x7f368c0cc4d0 2023-06-23T18:35:37.687 INFO:teuthology.orchestra.run.smithi155.stdout:7 2023-06-23T18:35:37.688 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.683+0000 7f36912c0700 1 -- 172.21.15.155:0/3488651760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f368c0cc4d0 msgr2=0x7f368c154f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.688 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.683+0000 7f36912c0700 1 --2- 172.21.15.155:0/3488651760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f368c0cc4d0 0x7f368c154f80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f368000c4e0 tx=0x7f368000c7f0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.688 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.683+0000 7f36912c0700 1 -- 172.21.15.155:0/3488651760 shutdown_connections 2023-06-23T18:35:37.688 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.683+0000 7f36912c0700 1 --2- 172.21.15.155:0/3488651760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f368c156ae0 0x7f368c1e44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.688 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.683+0000 7f36912c0700 1 --2- 172.21.15.155:0/3488651760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f368c0cc060 0x7f368c0b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.688 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.683+0000 7f36912c0700 1 --2- 172.21.15.155:0/3488651760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f368c0cc4d0 0x7f368c154f80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.689 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.683+0000 7f36912c0700 1 -- 172.21.15.155:0/3488651760 >> 172.21.15.155:0/3488651760 conn(0x7f368c06afa0 msgr2=0x7f368c0cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:37.689 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.683+0000 7f36912c0700 1 -- 172.21.15.155:0/3488651760 shutdown_connections 2023-06-23T18:35:37.689 INFO:teuthology.orchestra.run.smithi155.stderr:2023-06-23T18:35:37.683+0000 7f36912c0700 1 -- 172.21.15.155:0/3488651760 wait complete. 2023-06-23T18:35:37.699 INFO:tasks.ceph.osd.0:Restarting daemon 2023-06-23T18:35:37.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-06-23T18:35:37.702 INFO:tasks.ceph.osd.0:Started 2023-06-23T18:35:37.702 INFO:tasks.ceph.osd.1:Restarting daemon 2023-06-23T18:35:37.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-06-23T18:35:37.704 INFO:tasks.ceph.osd.1:Started 2023-06-23T18:35:37.705 INFO:tasks.ceph.osd.2:Restarting daemon 2023-06-23T18:35:37.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-06-23T18:35:37.707 INFO:tasks.ceph.osd.2:Started 2023-06-23T18:35:37.707 INFO:tasks.ceph.osd.3:Restarting daemon 2023-06-23T18:35:37.708 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-06-23T18:35:37.710 INFO:tasks.ceph.osd.3:Started 2023-06-23T18:35:37.710 INFO:tasks.ceph.osd.4:Restarting daemon 2023-06-23T18:35:37.710 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-06-23T18:35:37.712 INFO:tasks.ceph.osd.4:Started 2023-06-23T18:35:37.713 INFO:tasks.ceph.osd.5:Restarting daemon 2023-06-23T18:35:37.713 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-06-23T18:35:37.714 INFO:tasks.ceph.osd.5:Started 2023-06-23T18:35:37.715 INFO:tasks.ceph.osd.6:Restarting daemon 2023-06-23T18:35:37.715 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-06-23T18:35:37.717 INFO:tasks.ceph.osd.6:Started 2023-06-23T18:35:37.717 INFO:tasks.ceph.osd.7:Restarting daemon 2023-06-23T18:35:37.717 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-06-23T18:35:37.719 INFO:tasks.ceph.osd.7:Started 2023-06-23T18:35:37.720 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:35:37.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.861+0000 7f0cb4b01700 1 Processor -- start 2023-06-23T18:35:37.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.861+0000 7f0cb4b01700 1 -- start start 2023-06-23T18:35:37.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cb4b01700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0cb00cec00 0x7f0cb00b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cb4b01700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb00b1d40 con 0x7f0cb015a380 2023-06-23T18:35:37.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cb4b01700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb00b2020 con 0x7f0cb015a750 2023-06-23T18:35:37.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cb4b01700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb00ad650 con 0x7f0cb00cec00 2023-06-23T18:35:37.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cadd9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0cb00cec00 0x7f0cb00b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cae59c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f0cb015a380 0x7f0cb00ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41498/0 (socket says 172.21.15.79:41498) 2023-06-23T18:35:37.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cadd9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0cb00cec00 0x7f0cb00b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38458/0 (socket says 172.21.15.79:38458) 2023-06-23T18:35:37.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cae59c700 1 -- 172.21.15.79:0/2137177915 learned_addr learned my addr 172.21.15.79:0/2137177915 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:37.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cadd9b700 1 -- 172.21.15.79:0/2137177915 >> v1:172.21.15.155:6789/0 conn(0x7f0cb015a750 legacy=0x7f0cb00b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cadd9b700 1 -- 172.21.15.79:0/2137177915 >> v1:172.21.15.79:6789/0 conn(0x7f0cb015a380 legacy=0x7f0cb00ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cad59a700 1 -- 172.21.15.79:0/2137177915 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 788050115 0 0) 0x7f0cb00b1d40 con 0x7f0cb015a380 2023-06-23T18:35:37.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cadd9b700 1 -- 172.21.15.79:0/2137177915 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb00ad930 con 0x7f0cb00cec00 2023-06-23T18:35:37.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cadd9b700 1 --2- 172.21.15.79:0/2137177915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cb00cec00 0x7f0cb00b0000 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009840 tx=0x7f0ca4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3af070b61355f44 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:37.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cad59a700 1 -- 172.21.15.79:0/2137177915 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4011830 con 0x7f0cb00cec00 2023-06-23T18:35:37.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cad59a700 1 -- 172.21.15.79:0/2137177915 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca4011990 con 0x7f0cb00cec00 2023-06-23T18:35:37.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.863+0000 7f0cad59a700 1 -- 172.21.15.79:0/2137177915 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4011d20 con 0x7f0cb00cec00 2023-06-23T18:35:37.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.864+0000 7f0cb4b01700 1 -- 172.21.15.79:0/2137177915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cb00cec00 msgr2=0x7f0cb00b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.864+0000 7f0cb4b01700 1 --2- 172.21.15.79:0/2137177915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cb00cec00 0x7f0cb00b0000 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009840 tx=0x7f0ca4009b50 comp rx=0 tx=0).stop 2023-06-23T18:35:37.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.864+0000 7f0cb4b01700 1 -- 172.21.15.79:0/2137177915 shutdown_connections 2023-06-23T18:35:37.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.864+0000 7f0cb4b01700 1 --2- 172.21.15.79:0/2137177915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cb00cec00 0x7f0cb00b0000 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.864+0000 7f0cb4b01700 1 -- 172.21.15.79:0/2137177915 >> 172.21.15.79:0/2137177915 conn(0x7f0cb0089fc0 msgr2=0x7f0cb0099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:37.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.864+0000 7f0cb4b01700 1 -- 172.21.15.79:0/2137177915 shutdown_connections 2023-06-23T18:35:37.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.864+0000 7f0cb4b01700 1 -- 172.21.15.79:0/2137177915 wait complete. 2023-06-23T18:35:37.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.866+0000 7f0cb4b01700 1 Processor -- start 2023-06-23T18:35:37.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.866+0000 7f0cb4b01700 1 -- start start 2023-06-23T18:35:37.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cb4b01700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb00cec00 0x7f0cb006f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cb4b01700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cb015a380 0x7f0cb006fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cb4b01700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb015a750 0x7f0cb00b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:37.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cb4b01700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb00c5750 con 0x7f0cb015a380 2023-06-23T18:35:37.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cb4b01700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb00c4b60 con 0x7f0cb015a750 2023-06-23T18:35:37.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cb4b01700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb00b66f0 con 0x7f0cb00cec00 2023-06-23T18:35:37.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cadd9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cb015a380 0x7f0cb006fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cadd9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cb015a380 0x7f0cb006fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38464/0 (socket says 172.21.15.79:38464) 2023-06-23T18:35:37.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cadd9b700 1 -- 172.21.15.79:0/3832975638 learned_addr learned my addr 172.21.15.79:0/3832975638 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:37.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cae59c700 1 --2- 172.21.15.79:0/3832975638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb00cec00 0x7f0cb006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0caed9d700 1 --2- 172.21.15.79:0/3832975638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb015a750 0x7f0cb00b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:37.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cadd9b700 1 -- 172.21.15.79:0/3832975638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb00cec00 msgr2=0x7f0cb006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cadd9b700 1 --2- 172.21.15.79:0/3832975638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb00cec00 0x7f0cb006f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cadd9b700 1 -- 172.21.15.79:0/3832975638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb015a750 msgr2=0x7f0cb00b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cadd9b700 1 --2- 172.21.15.79:0/3832975638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb015a750 0x7f0cb00b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cadd9b700 1 -- 172.21.15.79:0/3832975638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca40094a0 con 0x7f0cb015a380 2023-06-23T18:35:37.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0cadd9b700 1 --2- 172.21.15.79:0/3832975638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cb015a380 0x7f0cb006fc70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40090b0 tx=0x7f0ca4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:37.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.867+0000 7f0c9effd700 1 -- 172.21.15.79:0/3832975638 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca401a860 con 0x7f0cb015a380 2023-06-23T18:35:37.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.868+0000 7f0c9effd700 1 -- 172.21.15.79:0/3832975638 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca401a9e0 con 0x7f0cb015a380 2023-06-23T18:35:37.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.868+0000 7f0cb4b01700 1 -- 172.21.15.79:0/3832975638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb00c4db0 con 0x7f0cb015a380 2023-06-23T18:35:37.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.868+0000 7f0c9effd700 1 -- 172.21.15.79:0/3832975638 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca401ad90 con 0x7f0cb015a380 2023-06-23T18:35:37.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.868+0000 7f0cb4b01700 1 -- 172.21.15.79:0/3832975638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb00c5370 con 0x7f0cb015a380 2023-06-23T18:35:37.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.868+0000 7f0c9effd700 1 -- 172.21.15.79:0/3832975638 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0ca4022070 con 0x7f0cb015a380 2023-06-23T18:35:37.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.868+0000 7f0c9effd700 1 -- 172.21.15.79:0/3832975638 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f0ca4024af0 con 0x7f0cb015a380 2023-06-23T18:35:37.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.868+0000 7f0cb4b01700 1 -- 172.21.15.79:0/3832975638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb00c5890 con 0x7f0cb015a380 2023-06-23T18:35:37.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.871+0000 7f0c9effd700 1 -- 172.21.15.79:0/3832975638 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f0cb00c5890 con 0x7f0cb015a380 2023-06-23T18:35:37.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.952+0000 7f0cb4b01700 1 -- 172.21.15.79:0/3832975638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0cb0099df0 con 0x7f0cb015a380 2023-06-23T18:35:37.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.952+0000 7f0c9effd700 1 -- 172.21.15.79:0/3832975638 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0ca4029070 con 0x7f0cb015a380 2023-06-23T18:35:37.958 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:37.958 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":10,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:37.676106+0000","last_up_change":"0.000000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:35:37.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.953+0000 7f0cb4b01700 1 -- 172.21.15.79:0/3832975638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cb015a380 msgr2=0x7f0cb006fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:37.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.953+0000 7f0cb4b01700 1 --2- 172.21.15.79:0/3832975638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cb015a380 0x7f0cb006fc70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40090b0 tx=0x7f0ca4004750 comp rx=0 tx=0).stop 2023-06-23T18:35:37.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.953+0000 7f0cb4b01700 1 -- 172.21.15.79:0/3832975638 shutdown_connections 2023-06-23T18:35:37.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.953+0000 7f0cb4b01700 1 --2- 172.21.15.79:0/3832975638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb015a750 0x7f0cb00b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.953+0000 7f0cb4b01700 1 --2- 172.21.15.79:0/3832975638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb00cec00 0x7f0cb006f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.953+0000 7f0cb4b01700 1 --2- 172.21.15.79:0/3832975638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cb015a380 0x7f0cb006fc70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:37.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.953+0000 7f0cb4b01700 1 -- 172.21.15.79:0/3832975638 >> 172.21.15.79:0/3832975638 conn(0x7f0cb0089fc0 msgr2=0x7f0cb00b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:37.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.953+0000 7f0cb4b01700 1 -- 172.21.15.79:0/3832975638 shutdown_connections 2023-06-23T18:35:37.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:37.954+0000 7f0cb4b01700 1 -- 172.21.15.79:0/3832975638 wait complete. 2023-06-23T18:35:37.970 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-06-23T18:35:37.971 INFO:tasks.ceph:Waiting for OSDs to come up 2023-06-23T18:35:38.078 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-06-23T18:35:38.073+0000 7f44ad0fa640 -1 Falling back to public interface 2023-06-23T18:35:38.201 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:38.196+0000 7f4a314e2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:35:38.279 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:38.274+0000 7f4a314e2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T18:35:38.283 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:38.279+0000 7fb8a0753200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:35:38.328 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-06-23T18:35:38.323+0000 7fb4cfece640 -1 Falling back to public interface 2023-06-23T18:35:38.332 INFO:tasks.ceph.osd.5.smithi155.stderr:2023-06-23T18:35:38.327+0000 7f86fcb21640 -1 Falling back to public interface 2023-06-23T18:35:38.332 INFO:tasks.ceph.osd.6.smithi155.stderr:2023-06-23T18:35:38.327+0000 7f2815303640 -1 Falling back to public interface 2023-06-23T18:35:38.333 INFO:tasks.ceph.osd.5.smithi155.stderr:2023-06-23T18:35:38.327+0000 7f86f3edf700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6801/0: (98) Address already in use 2023-06-23T18:35:38.333 INFO:tasks.ceph.osd.5.smithi155.stderr:2023-06-23T18:35:38.327+0000 7f86f3edf700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6802/0: (98) Address already in use 2023-06-23T18:35:38.333 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-06-23T18:35:38.328+0000 7fce5b974640 -1 Falling back to public interface 2023-06-23T18:35:38.337 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-23T18:35:38.332+0000 7f660422f640 -1 Falling back to public interface 2023-06-23T18:35:38.338 INFO:tasks.ceph.osd.4.smithi155.stderr:2023-06-23T18:35:38.333+0000 7f2c93146640 -1 Falling back to public interface 2023-06-23T18:35:38.341 INFO:tasks.ceph.osd.7.smithi155.stderr:2023-06-23T18:35:38.337+0000 7f21779c6640 -1 Falling back to public interface 2023-06-23T18:35:38.365 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:38.360+0000 7fb8a0753200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T18:35:38.420 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:38.415+0000 7f4a314e2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:35:38.503 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:38.498+0000 7fb8a0753200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:35:38.652 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:38.647+0000 7f4a314e2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T18:35:38.725 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:38.720+0000 7f4a314e2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T18:35:38.735 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:38.731+0000 7fb8a0753200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T18:35:38.775 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-23T18:35:38.809 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:38.804+0000 7fb8a0753200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T18:35:38.892 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.887+0000 7f28197fd700 1 Processor -- start 2023-06-23T18:35:38.892 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.887+0000 7f28197fd700 1 -- start start 2023-06-23T18:35:38.894 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.888+0000 7f28197fd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f281415c810 0x7f28140af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:38.894 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.888+0000 7f28197fd700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28140cf090 con 0x7f28140cf520 2023-06-23T18:35:38.894 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.888+0000 7f28197fd700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28140ce4f0 con 0x7f28140b01c0 2023-06-23T18:35:38.895 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.888+0000 7f28197fd700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f281415cbe0 con 0x7f281415c810 2023-06-23T18:35:38.895 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.888+0000 7f2812ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f281415c810 0x7f28140af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:38.895 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.888+0000 7f28127fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f28140cf520 0x7f28140ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41510/0 (socket says 172.21.15.79:41510) 2023-06-23T18:35:38.895 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.888+0000 7f2812ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f281415c810 0x7f28140af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38468/0 (socket says 172.21.15.79:38468) 2023-06-23T18:35:38.895 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.888+0000 7f28127fc700 1 -- 172.21.15.79:0/1322340628 learned_addr learned my addr 172.21.15.79:0/1322340628 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:38.895 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.888+0000 7f2812ffd700 1 -- 172.21.15.79:0/1322340628 >> v1:172.21.15.155:6789/0 conn(0x7f28140b01c0 legacy=0x7f28140b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:38.895 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.888+0000 7f2812ffd700 1 -- 172.21.15.79:0/1322340628 >> v1:172.21.15.79:6789/0 conn(0x7f28140cf520 legacy=0x7f28140ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:38.895 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.888+0000 7f2812ffd700 1 -- 172.21.15.79:0/1322340628 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28140ce760 con 0x7f281415c810 2023-06-23T18:35:38.896 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.888+0000 7f2811ffb700 1 -- 172.21.15.79:0/1322340628 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2989331065 0 0) 0x7f28140cf090 con 0x7f28140cf520 2023-06-23T18:35:38.896 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.889+0000 7f2812ffd700 1 --2- 172.21.15.79:0/1322340628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f281415c810 0x7f28140af630 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2804009840 tx=0x7f2804009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ad5e2bbd5135b9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:38.896 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.889+0000 7f2811ffb700 1 -- 172.21.15.79:0/1322340628 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804011830 con 0x7f281415c810 2023-06-23T18:35:38.896 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.889+0000 7f2811ffb700 1 -- 172.21.15.79:0/1322340628 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2804011990 con 0x7f281415c810 2023-06-23T18:35:38.896 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.889+0000 7f2811ffb700 1 -- 172.21.15.79:0/1322340628 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804011d20 con 0x7f281415c810 2023-06-23T18:35:38.896 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.889+0000 7f28197fd700 1 -- 172.21.15.79:0/1322340628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f281415c810 msgr2=0x7f28140af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:38.897 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.889+0000 7f28197fd700 1 --2- 172.21.15.79:0/1322340628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f281415c810 0x7f28140af630 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2804009840 tx=0x7f2804009b50 comp rx=0 tx=0).stop 2023-06-23T18:35:38.897 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.889+0000 7f28197fd700 1 -- 172.21.15.79:0/1322340628 shutdown_connections 2023-06-23T18:35:38.897 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.889+0000 7f28197fd700 1 --2- 172.21.15.79:0/1322340628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f281415c810 0x7f28140af630 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:38.897 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.889+0000 7f28197fd700 1 -- 172.21.15.79:0/1322340628 >> 172.21.15.79:0/1322340628 conn(0x7f281406b920 msgr2=0x7f28140cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:38.897 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.889+0000 7f28197fd700 1 -- 172.21.15.79:0/1322340628 shutdown_connections 2023-06-23T18:35:38.898 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.889+0000 7f28197fd700 1 -- 172.21.15.79:0/1322340628 wait complete. 2023-06-23T18:35:38.901 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.891+0000 7f28197fd700 1 Processor -- start 2023-06-23T18:35:38.901 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.891+0000 7f28197fd700 1 -- start start 2023-06-23T18:35:38.901 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f28197fd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28140b01c0 0x7f28140ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:38.901 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f28197fd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28140cf520 0x7f28140b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:38.901 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f28197fd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f281415c810 0x7f281415bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:38.901 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f28197fd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28140adde0 con 0x7f28140b01c0 2023-06-23T18:35:38.902 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f28197fd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28140ae0c0 con 0x7f281415c810 2023-06-23T18:35:38.902 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f28197fd700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28140ae3a0 con 0x7f28140cf520 2023-06-23T18:35:38.902 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f2812ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28140b01c0 0x7f28140ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:38.902 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f2812ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28140b01c0 0x7f28140ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38474/0 (socket says 172.21.15.79:38474) 2023-06-23T18:35:38.902 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f2812ffd700 1 -- 172.21.15.79:0/3948560185 learned_addr learned my addr 172.21.15.79:0/3948560185 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:38.902 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f28127fc700 1 --2- 172.21.15.79:0/3948560185 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28140cf520 0x7f28140b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:38.902 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f28137fe700 1 --2- 172.21.15.79:0/3948560185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f281415c810 0x7f281415bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:38.903 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f2812ffd700 1 -- 172.21.15.79:0/3948560185 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28140cf520 msgr2=0x7f28140b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:38.903 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f2812ffd700 1 --2- 172.21.15.79:0/3948560185 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28140cf520 0x7f28140b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:38.903 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f2812ffd700 1 -- 172.21.15.79:0/3948560185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f281415c810 msgr2=0x7f281415bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:38.903 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f2812ffd700 1 --2- 172.21.15.79:0/3948560185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f281415c810 0x7f281415bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:38.903 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f2812ffd700 1 -- 172.21.15.79:0/3948560185 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28040094a0 con 0x7f28140b01c0 2023-06-23T18:35:38.903 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f28137fe700 1 --2- 172.21.15.79:0/3948560185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f281415c810 0x7f281415bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:35:38.903 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.892+0000 7f28127fc700 1 --2- 172.21.15.79:0/3948560185 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28140cf520 0x7f28140b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:35:38.903 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.893+0000 7f2812ffd700 1 --2- 172.21.15.79:0/3948560185 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28140b01c0 0x7f28140ad6e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2804014040 tx=0x7f2804004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:38.904 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.893+0000 7f27fb7fe700 1 -- 172.21.15.79:0/3948560185 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280401a980 con 0x7f28140b01c0 2023-06-23T18:35:38.904 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.893+0000 7f27fb7fe700 1 -- 172.21.15.79:0/3948560185 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280401ab00 con 0x7f28140b01c0 2023-06-23T18:35:38.904 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.893+0000 7f28197fd700 1 -- 172.21.15.79:0/3948560185 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28140ae680 con 0x7f28140b01c0 2023-06-23T18:35:38.904 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.893+0000 7f27fb7fe700 1 -- 172.21.15.79:0/3948560185 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804024420 con 0x7f28140b01c0 2023-06-23T18:35:38.904 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.893+0000 7f28197fd700 1 -- 172.21.15.79:0/3948560185 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281415c240 con 0x7f28140b01c0 2023-06-23T18:35:38.904 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.893+0000 7f27fb7fe700 1 -- 172.21.15.79:0/3948560185 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2804022070 con 0x7f28140b01c0 2023-06-23T18:35:38.904 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.893+0000 7f27fb7fe700 1 -- 172.21.15.79:0/3948560185 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f280402da80 con 0x7f28140b01c0 2023-06-23T18:35:38.905 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.894+0000 7f28197fd700 1 -- 172.21.15.79:0/3948560185 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2814067a30 con 0x7f28140b01c0 2023-06-23T18:35:38.905 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.896+0000 7f27fb7fe700 1 -- 172.21.15.79:0/3948560185 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f2804029070 con 0x7f28140b01c0 2023-06-23T18:35:38.970 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.965+0000 7f28197fd700 1 -- 172.21.15.79:0/3948560185 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f281415c3d0 con 0x7f28140b01c0 2023-06-23T18:35:38.971 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.966+0000 7f27fb7fe700 1 -- 172.21.15.79:0/3948560185 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2804015020 con 0x7f28140b01c0 2023-06-23T18:35:38.971 INFO:teuthology.misc.health.smithi079.stdout: 2023-06-23T18:35:38.971 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":10,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:37.676106+0000","last_up_change":"0.000000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:35:38.972 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.967+0000 7f28197fd700 1 -- 172.21.15.79:0/3948560185 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28140b01c0 msgr2=0x7f28140ad6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:38.973 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.967+0000 7f28197fd700 1 --2- 172.21.15.79:0/3948560185 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28140b01c0 0x7f28140ad6e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2804014040 tx=0x7f2804004750 comp rx=0 tx=0).stop 2023-06-23T18:35:38.973 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.967+0000 7f28197fd700 1 -- 172.21.15.79:0/3948560185 shutdown_connections 2023-06-23T18:35:38.973 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.967+0000 7f28197fd700 1 --2- 172.21.15.79:0/3948560185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f281415c810 0x7f281415bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:38.973 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.967+0000 7f28197fd700 1 --2- 172.21.15.79:0/3948560185 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28140b01c0 0x7f28140ad6e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:38.973 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.967+0000 7f28197fd700 1 --2- 172.21.15.79:0/3948560185 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28140cf520 0x7f28140b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:38.973 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.967+0000 7f28197fd700 1 -- 172.21.15.79:0/3948560185 >> 172.21.15.79:0/3948560185 conn(0x7f281406b920 msgr2=0x7f281406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:38.973 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.967+0000 7f28197fd700 1 -- 172.21.15.79:0/3948560185 shutdown_connections 2023-06-23T18:35:38.973 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:38.967+0000 7f28197fd700 1 -- 172.21.15.79:0/3948560185 wait complete. 2023-06-23T18:35:38.982 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-06-23T18:35:39.004 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:39.000+0000 7f4a314e2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T18:35:39.081 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:39.076+0000 7fb8a0753200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T18:35:39.221 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:39.217+0000 7f4a314e2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T18:35:39.298 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:39.294+0000 7fb8a0753200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T18:35:39.376 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:39.370+0000 7f4a314e2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T18:35:39.452 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:39.448+0000 7fb8a0753200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T18:35:39.490 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:39.485+0000 7f4a314e2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T18:35:39.565 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:39.561+0000 7fb8a0753200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T18:35:39.637 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:39.632+0000 7f4a314e2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T18:35:39.714 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:39.710+0000 7fb8a0753200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T18:35:39.907 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:39.902+0000 7f4a314e2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T18:35:39.993 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:39.988+0000 7fb8a0753200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T18:35:40.090 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:40.086+0000 7f4a314e2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T18:35:40.177 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:40.172+0000 7fb8a0753200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T18:35:40.678 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:40.673+0000 7f4a314e2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T18:35:40.766 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-06-23T18:35:40.761+0000 7fb8a0753200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T18:35:41.564 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-06-23T18:35:41.559+0000 7f44ad0fa640 -1 osd.1 0 log_to_monitors true 2023-06-23T18:35:41.702 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:41.698+0000 7f49e970e700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-06-23T18:35:41.704 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-23T18:35:41.699+0000 7f49e970e700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-06-23T18:35:41.834 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-06-23T18:35:41.829+0000 7fb4cfece640 -1 osd.0 0 log_to_monitors true 2023-06-23T18:35:41.856 INFO:tasks.ceph.osd.6.smithi155.stderr:2023-06-23T18:35:41.851+0000 7f2815303640 -1 osd.6 0 log_to_monitors true 2023-06-23T18:35:41.868 INFO:tasks.ceph.osd.5.smithi155.stderr:2023-06-23T18:35:41.864+0000 7f86fcb21640 -1 osd.5 0 log_to_monitors true 2023-06-23T18:35:41.877 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-23T18:35:41.872+0000 7f660422f640 -1 osd.3 0 log_to_monitors true 2023-06-23T18:35:41.886 INFO:tasks.ceph.osd.4.smithi155.stderr:2023-06-23T18:35:41.882+0000 7f2c93146640 -1 osd.4 0 log_to_monitors true 2023-06-23T18:35:41.890 INFO:tasks.ceph.osd.7.smithi155.stderr:2023-06-23T18:35:41.886+0000 7f21779c6640 -1 osd.7 0 log_to_monitors true 2023-06-23T18:35:41.898 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-06-23T18:35:41.893+0000 7fce5b974640 -1 osd.2 0 log_to_monitors true 2023-06-23T18:35:44.905 INFO:tasks.ceph.osd.4.smithi155.stderr:2023-06-23T18:35:44.899+0000 7f2c86cfd700 -1 osd.4 0 waiting for initial osdmap 2023-06-23T18:35:44.919 INFO:tasks.ceph.osd.4.smithi155.stderr:2023-06-23T18:35:44.914+0000 7f2c7f0d8700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:35:44.924 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-23T18:35:44.919+0000 7f65f7de6700 -1 osd.3 0 waiting for initial osdmap 2023-06-23T18:35:44.937 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-23T18:35:44.933+0000 7f65f01c1700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:35:44.960 INFO:tasks.ceph.osd.7.smithi155.stderr:2023-06-23T18:35:44.955+0000 7f216b57d700 -1 osd.7 0 waiting for initial osdmap 2023-06-23T18:35:44.967 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-06-23T18:35:44.962+0000 7fb4c3a85700 -1 osd.0 0 waiting for initial osdmap 2023-06-23T18:35:44.972 INFO:tasks.ceph.osd.6.smithi155.stderr:2023-06-23T18:35:44.966+0000 7f2808eba700 -1 osd.6 0 waiting for initial osdmap 2023-06-23T18:35:44.973 INFO:tasks.ceph.osd.7.smithi155.stderr:2023-06-23T18:35:44.968+0000 7f2163958700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:35:44.980 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-06-23T18:35:44.975+0000 7fb4bbe60700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:35:44.988 INFO:tasks.ceph.osd.5.smithi155.stderr:2023-06-23T18:35:44.982+0000 7f86f06d8700 -1 osd.5 0 waiting for initial osdmap 2023-06-23T18:35:44.988 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-06-23T18:35:44.983+0000 7fce4f52b700 -1 osd.2 0 waiting for initial osdmap 2023-06-23T18:35:44.991 INFO:tasks.ceph.osd.6.smithi155.stderr:2023-06-23T18:35:44.986+0000 7f2801295700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:35:44.993 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-06-23T18:35:44.988+0000 7f44a0cb1700 -1 osd.1 0 waiting for initial osdmap 2023-06-23T18:35:45.001 INFO:tasks.ceph.osd.5.smithi155.stderr:2023-06-23T18:35:44.996+0000 7f86e8ab3700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:35:45.002 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-06-23T18:35:44.997+0000 7fce47906700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:35:45.004 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-06-23T18:35:45.000+0000 7f449908c700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:35:45.788 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-23T18:35:45.910 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.905+0000 7f3cfb763700 1 Processor -- start 2023-06-23T18:35:45.910 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.905+0000 7f3cfb763700 1 -- start start 2023-06-23T18:35:45.911 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.906+0000 7f3cfb763700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3cf40caf60 0x7f3cf4069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:45.911 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.906+0000 7f3cfb763700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cf40cb330 0x7f3cf4070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:45.912 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.906+0000 7f3cfb763700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf406b2a0 con 0x7f3cf4071120 2023-06-23T18:35:45.912 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.906+0000 7f3cfb763700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cf40715a0 con 0x7f3cf40caf60 2023-06-23T18:35:45.912 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.906+0000 7f3cfb763700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf406a430 con 0x7f3cf40cb330 2023-06-23T18:35:45.912 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.906+0000 7f3cf94ff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3cf40caf60 0x7f3cf4069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:45.912 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.906+0000 7f3cf94ff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3cf40caf60 0x7f3cf4069ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38580/0 (socket says 172.21.15.79:38580) 2023-06-23T18:35:45.912 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.906+0000 7f3cf94ff700 1 -- 172.21.15.79:0/1372775610 learned_addr learned my addr 172.21.15.79:0/1372775610 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:45.913 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.906+0000 7f3cf8cfe700 1 --2- 172.21.15.79:0/1372775610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cf40cb330 0x7f3cf4070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:45.913 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.906+0000 7f3cf94ff700 1 -- 172.21.15.79:0/1372775610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cf40cb330 msgr2=0x7f3cf4070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:45.913 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.906+0000 7f3cf94ff700 1 --2- 172.21.15.79:0/1372775610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cf40cb330 0x7f3cf4070be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:45.913 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.906+0000 7f3cf94ff700 1 -- 172.21.15.79:0/1372775610 >> v1:172.21.15.155:6789/0 conn(0x7f3cf4071120 legacy=0x7f3cf415dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:45.913 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.906+0000 7f3cf94ff700 1 -- 172.21.15.79:0/1372775610 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf40cdc10 con 0x7f3cf40caf60 2023-06-23T18:35:45.913 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.906+0000 7f3cf94ff700 1 --2- 172.21.15.79:0/1372775610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf40caf60 0x7f3cf4069ef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ce40098f0 tx=0x7f3ce4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44edac3e0bd8d139 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:45.913 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.907+0000 7f3cebfff700 1 -- 172.21.15.79:0/1372775610 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4004230 con 0x7f3cf40caf60 2023-06-23T18:35:45.913 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.907+0000 7f3cebfff700 1 -- 172.21.15.79:0/1372775610 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce4004390 con 0x7f3cf40caf60 2023-06-23T18:35:45.914 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.907+0000 7f3cebfff700 1 -- 172.21.15.79:0/1372775610 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce40036a0 con 0x7f3cf40caf60 2023-06-23T18:35:45.916 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.907+0000 7f3cfb763700 1 -- 172.21.15.79:0/1372775610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf40caf60 msgr2=0x7f3cf4069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:45.916 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.907+0000 7f3cfb763700 1 --2- 172.21.15.79:0/1372775610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf40caf60 0x7f3cf4069ef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ce40098f0 tx=0x7f3ce4009c00 comp rx=0 tx=0).stop 2023-06-23T18:35:45.916 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.907+0000 7f3cfb763700 1 -- 172.21.15.79:0/1372775610 shutdown_connections 2023-06-23T18:35:45.916 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.907+0000 7f3cfb763700 1 --2- 172.21.15.79:0/1372775610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf40caf60 0x7f3cf4069ef0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:45.916 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.907+0000 7f3cfb763700 1 --2- 172.21.15.79:0/1372775610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cf40cb330 0x7f3cf4070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:45.916 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.907+0000 7f3cfb763700 1 -- 172.21.15.79:0/1372775610 >> 172.21.15.79:0/1372775610 conn(0x7f3cf40cf030 msgr2=0x7f3cf40cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:45.917 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.907+0000 7f3cfb763700 1 -- 172.21.15.79:0/1372775610 shutdown_connections 2023-06-23T18:35:45.917 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.907+0000 7f3cfb763700 1 -- 172.21.15.79:0/1372775610 wait complete. 2023-06-23T18:35:45.917 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.908+0000 7f3cfb763700 1 Processor -- start 2023-06-23T18:35:45.917 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.908+0000 7f3cfb763700 1 -- start start 2023-06-23T18:35:45.917 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.909+0000 7f3cfb763700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4071120 0x7f3cf40b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:45.917 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.909+0000 7f3cfb763700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cf40caf60 0x7f3cf40c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:45.917 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.909+0000 7f3cfb763700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf40cb330 0x7f3cf409ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:45.918 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.909+0000 7f3cfb763700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf40b4ba0 con 0x7f3cf4071120 2023-06-23T18:35:45.918 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.909+0000 7f3cfb763700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf40c8470 con 0x7f3cf40cb330 2023-06-23T18:35:45.918 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.909+0000 7f3cfb763700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf40c8750 con 0x7f3cf40caf60 2023-06-23T18:35:45.918 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.909+0000 7f3cf94ff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4071120 0x7f3cf40b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:45.918 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.909+0000 7f3cf94ff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4071120 0x7f3cf40b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38586/0 (socket says 172.21.15.79:38586) 2023-06-23T18:35:45.918 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.909+0000 7f3cf94ff700 1 -- 172.21.15.79:0/729989734 learned_addr learned my addr 172.21.15.79:0/729989734 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:45.918 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.909+0000 7f3cf94ff700 1 -- 172.21.15.79:0/729989734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cf40caf60 msgr2=0x7f3cf40c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:45.919 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.909+0000 7f3cf94ff700 1 --2- 172.21.15.79:0/729989734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cf40caf60 0x7f3cf40c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:45.919 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.909+0000 7f3cf94ff700 1 -- 172.21.15.79:0/729989734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf40cb330 msgr2=0x7f3cf409ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:45.919 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.910+0000 7f3cf9d00700 1 --2- 172.21.15.79:0/729989734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf40cb330 0x7f3cf409ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:45.919 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.910+0000 7f3cf94ff700 1 --2- 172.21.15.79:0/729989734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf40cb330 0x7f3cf409ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:45.919 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.910+0000 7f3cf94ff700 1 -- 172.21.15.79:0/729989734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce4009550 con 0x7f3cf4071120 2023-06-23T18:35:45.920 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.910+0000 7f3cf9d00700 1 --2- 172.21.15.79:0/729989734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf40cb330 0x7f3cf409ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:35:45.920 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.910+0000 7f3cf94ff700 1 --2- 172.21.15.79:0/729989734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4071120 0x7f3cf40b12c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009130 tx=0x7f3ce4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:45.920 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.910+0000 7f3ce9ffb700 1 -- 172.21.15.79:0/729989734 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4003e10 con 0x7f3cf4071120 2023-06-23T18:35:45.920 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.910+0000 7f3ce9ffb700 1 -- 172.21.15.79:0/729989734 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce400b660 con 0x7f3cf4071120 2023-06-23T18:35:45.920 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.910+0000 7f3cfb763700 1 -- 172.21.15.79:0/729989734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf40c8a30 con 0x7f3cf4071120 2023-06-23T18:35:45.920 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.910+0000 7f3ce9ffb700 1 -- 172.21.15.79:0/729989734 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce400ba10 con 0x7f3cf4071120 2023-06-23T18:35:45.920 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.910+0000 7f3cfb763700 1 -- 172.21.15.79:0/729989734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf40b1800 con 0x7f3cf4071120 2023-06-23T18:35:45.920 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.911+0000 7f3cfb763700 1 -- 172.21.15.79:0/729989734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf406cb10 con 0x7f3cf4071120 2023-06-23T18:35:45.921 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.911+0000 7f3ce9ffb700 1 -- 172.21.15.79:0/729989734 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f3ce400bb70 con 0x7f3cf4071120 2023-06-23T18:35:45.921 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.912+0000 7f3ce9ffb700 1 --2- 172.21.15.79:0/729989734 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3cd806ac50 0x7f3cd806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:45.922 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.912+0000 7f3ce9ffb700 1 -- 172.21.15.79:0/729989734 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f3ce4089cf0 con 0x7f3cf4071120 2023-06-23T18:35:45.922 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.912+0000 7f3cf8cfe700 1 --2- 172.21.15.79:0/729989734 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3cd806ac50 0x7f3cd806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:45.922 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.912+0000 7f3cf8cfe700 1 --2- 172.21.15.79:0/729989734 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3cd806ac50 0x7f3cd806d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00098d0 tx=0x7f3cf0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:45.922 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:45.916+0000 7f3ce9ffb700 1 -- 172.21.15.79:0/729989734 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3ce4055e00 con 0x7f3cf4071120 2023-06-23T18:35:46.120 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:46.116+0000 7f3cfb763700 1 -- 172.21.15.79:0/729989734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cf40741c0 con 0x7f3cf4071120 2023-06-23T18:35:46.121 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:46.116+0000 7f3ce9ffb700 1 -- 172.21.15.79:0/729989734 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10920 (secure 0 0 0) 0x7f3ce401f090 con 0x7f3cf4071120 2023-06-23T18:35:46.121 INFO:teuthology.misc.health.smithi079.stdout: 2023-06-23T18:35:46.122 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":13,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:45.927394+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:35:46.123 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:46.119+0000 7f3cfb763700 1 -- 172.21.15.79:0/729989734 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3cd806ac50 msgr2=0x7f3cd806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:46.124 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:46.119+0000 7f3cfb763700 1 --2- 172.21.15.79:0/729989734 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3cd806ac50 0x7f3cd806d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00098d0 tx=0x7f3cf0008040 comp rx=0 tx=0).stop 2023-06-23T18:35:46.124 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:46.119+0000 7f3cfb763700 1 -- 172.21.15.79:0/729989734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4071120 msgr2=0x7f3cf40b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:46.124 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:46.119+0000 7f3cfb763700 1 --2- 172.21.15.79:0/729989734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4071120 0x7f3cf40b12c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009130 tx=0x7f3ce4004750 comp rx=0 tx=0).stop 2023-06-23T18:35:46.124 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:46.119+0000 7f3cfb763700 1 -- 172.21.15.79:0/729989734 shutdown_connections 2023-06-23T18:35:46.124 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:46.119+0000 7f3cfb763700 1 --2- 172.21.15.79:0/729989734 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3cd806ac50 0x7f3cd806d100 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:46.124 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:46.119+0000 7f3cfb763700 1 --2- 172.21.15.79:0/729989734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf40cb330 0x7f3cf409ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:46.125 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:46.119+0000 7f3cfb763700 1 --2- 172.21.15.79:0/729989734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4071120 0x7f3cf40b12c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:46.125 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:46.119+0000 7f3cfb763700 1 --2- 172.21.15.79:0/729989734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cf40caf60 0x7f3cf40c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:46.125 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:46.119+0000 7f3cfb763700 1 -- 172.21.15.79:0/729989734 >> 172.21.15.79:0/729989734 conn(0x7f3cf40cf030 msgr2=0x7f3cf40cde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:46.125 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:46.119+0000 7f3cfb763700 1 -- 172.21.15.79:0/729989734 shutdown_connections 2023-06-23T18:35:46.125 INFO:teuthology.misc.health.smithi079.stderr:2023-06-23T18:35:46.119+0000 7f3cfb763700 1 -- 172.21.15.79:0/729989734 wait complete. 2023-06-23T18:35:46.134 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-06-23T18:35:46.135 INFO:tasks.ceph:Creating RBD pool 2023-06-23T18:35:46.135 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd pool create rbd 8 2023-06-23T18:35:46.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.249+0000 7feece22b700 1 Processor -- start 2023-06-23T18:35:46.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.249+0000 7feece22b700 1 -- start start 2023-06-23T18:35:46.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feece22b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feec8156ae0 0x7feec815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:46.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feece22b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec80cc060 0x7feec80cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:46.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feece22b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec80cc9f0 con 0x7feec8071710 2023-06-23T18:35:46.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feece22b700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec815daf0 con 0x7feec8156ae0 2023-06-23T18:35:46.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feece22b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec815a560 con 0x7feec80cc060 2023-06-23T18:35:46.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec7fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7feec8071710 0x7feec8071b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41782/0 (socket says 172.21.15.79:41782) 2023-06-23T18:35:46.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec7fff700 1 -- 172.21.15.79:0/3582236489 learned_addr learned my addr 172.21.15.79:0/3582236489 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:46.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec77fe700 1 --2- 172.21.15.79:0/3582236489 >> v2:172.21.15.155:3300/0 conn(0x7feec8156ae0 0x7feec815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:46.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec6ffd700 1 --2- 172.21.15.79:0/3582236489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec80cc060 0x7feec80cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:46.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec67fc700 1 -- 172.21.15.79:0/3582236489 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942606612 0 0) 0x7feec0001490 con 0x7feec8071710 2023-06-23T18:35:46.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec67fc700 1 -- 172.21.15.79:0/3582236489 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeb40035b0 con 0x7feec8071710 2023-06-23T18:35:46.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec67fc700 1 -- 172.21.15.79:0/3582236489 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 45439280 0 0) 0x7feeb40035b0 con 0x7feec8071710 2023-06-23T18:35:46.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec67fc700 1 -- 172.21.15.79:0/3582236489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec80cc060 msgr2=0x7feec80cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:46.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec67fc700 1 --2- 172.21.15.79:0/3582236489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec80cc060 0x7feec80cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:46.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec67fc700 1 -- 172.21.15.79:0/3582236489 >> v2:172.21.15.155:3300/0 conn(0x7feec8156ae0 msgr2=0x7feec815d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:46.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec67fc700 1 --2- 172.21.15.79:0/3582236489 >> v2:172.21.15.155:3300/0 conn(0x7feec8156ae0 0x7feec815d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:46.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec67fc700 1 -- 172.21.15.79:0/3582236489 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec8159950 con 0x7feec8071710 2023-06-23T18:35:46.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec67fc700 1 -- 172.21.15.79:0/3582236489 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7feec0001b80 con 0x7feec8071710 2023-06-23T18:35:46.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec77fe700 1 --2- 172.21.15.79:0/3582236489 >> v2:172.21.15.155:3300/0 conn(0x7feec8156ae0 0x7feec815d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:35:46.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec67fc700 1 -- 172.21.15.79:0/3582236489 >> v1:172.21.15.79:6789/0 conn(0x7feec8071710 legacy=0x7feec8071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:46.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec67fc700 1 --2- 172.21.15.79:0/3582236489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec80cc060 0x7feeb4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:46.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.250+0000 7feec67fc700 1 --2- 172.21.15.79:0/3582236489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb4003f80 0x7feeb4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:46.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec67fc700 1 --2- 172.21.15.79:0/3582236489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feeb4004d90 0x7feeb4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:46.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec67fc700 1 -- 172.21.15.79:0/3582236489 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec815daf0 con 0x7feeb4004d90 2023-06-23T18:35:46.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec67fc700 1 -- 172.21.15.79:0/3582236489 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec815a560 con 0x7feeb4003f80 2023-06-23T18:35:46.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec67fc700 1 -- 172.21.15.79:0/3582236489 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb400ba40 con 0x7feec80cc060 2023-06-23T18:35:46.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec7fff700 1 --2- 172.21.15.79:0/3582236489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feeb4004d90 0x7feeb4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:46.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec77fe700 1 --2- 172.21.15.79:0/3582236489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec80cc060 0x7feeb4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:46.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec6ffd700 1 --2- 172.21.15.79:0/3582236489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb4003f80 0x7feeb4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:46.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec7fff700 1 -- 172.21.15.79:0/3582236489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec80cc060 msgr2=0x7feeb4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:46.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec7fff700 1 --2- 172.21.15.79:0/3582236489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec80cc060 0x7feeb4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:46.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec7fff700 1 -- 172.21.15.79:0/3582236489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb4003f80 msgr2=0x7feeb4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:46.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec7fff700 1 --2- 172.21.15.79:0/3582236489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb4003f80 0x7feeb4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:46.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec7fff700 1 -- 172.21.15.79:0/3582236489 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb40035b0 con 0x7feeb4004d90 2023-06-23T18:35:46.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec6ffd700 1 --2- 172.21.15.79:0/3582236489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb4003f80 0x7feeb4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:35:46.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec7fff700 1 --2- 172.21.15.79:0/3582236489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feeb4004d90 0x7feeb4005200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7feec000a830 tx=0x7feeb4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fc7e344594c3a3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:46.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec67fc700 1 -- 172.21.15.79:0/3582236489 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0012600 con 0x7feeb4004d90 2023-06-23T18:35:46.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec67fc700 1 -- 172.21.15.79:0/3582236489 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec00127a0 con 0x7feeb4004d90 2023-06-23T18:35:46.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.251+0000 7feec67fc700 1 -- 172.21.15.79:0/3582236489 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0012b50 con 0x7feeb4004d90 2023-06-23T18:35:46.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.252+0000 7feece22b700 1 -- 172.21.15.79:0/3582236489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feeb4004d90 msgr2=0x7feeb4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:46.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.252+0000 7feece22b700 1 --2- 172.21.15.79:0/3582236489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feeb4004d90 0x7feeb4005200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7feec000a830 tx=0x7feeb4005880 comp rx=0 tx=0).stop 2023-06-23T18:35:46.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.252+0000 7feec7fff700 1 -- 172.21.15.79:0/3582236489 reap_dead start 2023-06-23T18:35:46.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.252+0000 7feece22b700 1 -- 172.21.15.79:0/3582236489 shutdown_connections 2023-06-23T18:35:46.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.252+0000 7feece22b700 1 -- 172.21.15.79:0/3582236489 >> 172.21.15.79:0/3582236489 conn(0x7feec806afa0 msgr2=0x7feec806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:46.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.252+0000 7feece22b700 1 -- 172.21.15.79:0/3582236489 shutdown_connections 2023-06-23T18:35:46.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.252+0000 7feece22b700 1 -- 172.21.15.79:0/3582236489 wait complete. 2023-06-23T18:35:46.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.253+0000 7feece22b700 1 Processor -- start 2023-06-23T18:35:46.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.253+0000 7feece22b700 1 -- start start 2023-06-23T18:35:46.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feece22b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feec8149af0 0x7feec80cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:46.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feece22b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feec8154010 0x7feec8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:46.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feece22b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec806a3d0 0x7feec815a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:46.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feece22b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec806a850 con 0x7feec8154010 2023-06-23T18:35:46.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feece22b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec815c5a0 con 0x7feec8149af0 2023-06-23T18:35:46.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feece22b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec80ab8b0 con 0x7feec806a3d0 2023-06-23T18:35:46.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feec6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feec8154010 0x7feec8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:46.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feec6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feec8154010 0x7feec8069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38750/0 (socket says 172.21.15.79:38750) 2023-06-23T18:35:46.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feec6ffd700 1 -- 172.21.15.79:0/2491966552 learned_addr learned my addr 172.21.15.79:0/2491966552 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:46.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feec77fe700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feec8149af0 0x7feec80cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:46.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feec7fff700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec806a3d0 0x7feec815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:46.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feec6ffd700 1 -- 172.21.15.79:0/2491966552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec806a3d0 msgr2=0x7feec815a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:46.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feec6ffd700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec806a3d0 0x7feec815a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:46.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feec6ffd700 1 -- 172.21.15.79:0/2491966552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feec8149af0 msgr2=0x7feec80cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:46.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feec6ffd700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feec8149af0 0x7feec80cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:46.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feec6ffd700 1 -- 172.21.15.79:0/2491966552 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec815b7e0 con 0x7feec8154010 2023-06-23T18:35:46.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feec7fff700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec806a3d0 0x7feec815a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:35:46.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feec6ffd700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feec8154010 0x7feec8069e90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feeb8010850 tx=0x7feeb8010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:46.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feebffff700 1 -- 172.21.15.79:0/2491966552 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8010e50 con 0x7feec8154010 2023-06-23T18:35:46.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feebffff700 1 -- 172.21.15.79:0/2491966552 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb8004020 con 0x7feec8154010 2023-06-23T18:35:46.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feece22b700 1 -- 172.21.15.79:0/2491966552 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec815bac0 con 0x7feec8154010 2023-06-23T18:35:46.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feebffff700 1 -- 172.21.15.79:0/2491966552 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb80043b0 con 0x7feec8154010 2023-06-23T18:35:46.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.254+0000 7feece22b700 1 -- 172.21.15.79:0/2491966552 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec815c080 con 0x7feec8154010 2023-06-23T18:35:46.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.255+0000 7feece22b700 1 -- 172.21.15.79:0/2491966552 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec80672c0 con 0x7feec8154010 2023-06-23T18:35:46.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.256+0000 7feebffff700 1 -- 172.21.15.79:0/2491966552 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7feeb800e8d0 con 0x7feec8154010 2023-06-23T18:35:46.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.256+0000 7feebffff700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7feea806ac50 0x7feea806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:46.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.256+0000 7feec77fe700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7feea806ac50 0x7feea806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:46.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.256+0000 7feebffff700 1 -- 172.21.15.79:0/2491966552 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7feeb8088fb0 con 0x7feec8154010 2023-06-23T18:35:46.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.257+0000 7feec77fe700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7feea806ac50 0x7feea806d100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feec8159230 tx=0x7feeb0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:46.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.261+0000 7feebffff700 1 -- 172.21.15.79:0/2491966552 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feeb80bb310 con 0x7feec8154010 2023-06-23T18:35:46.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.449+0000 7feece22b700 1 -- 172.21.15.79:0/2491966552 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7feec80ce220 con 0x7feec8154010 2023-06-23T18:35:46.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.958+0000 7feebffff700 1 -- 172.21.15.79:0/2491966552 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7feeb8054800 con 0x7feec8154010 2023-06-23T18:35:46.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.961+0000 7feece22b700 1 -- 172.21.15.79:0/2491966552 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7feea806ac50 msgr2=0x7feea806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:46.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.961+0000 7feece22b700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7feea806ac50 0x7feea806d100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feec8159230 tx=0x7feeb0008040 comp rx=0 tx=0).stop 2023-06-23T18:35:46.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.961+0000 7feece22b700 1 -- 172.21.15.79:0/2491966552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feec8154010 msgr2=0x7feec8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:46.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.961+0000 7feece22b700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feec8154010 0x7feec8069e90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feeb8010850 tx=0x7feeb8010c10 comp rx=0 tx=0).stop 2023-06-23T18:35:46.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.961+0000 7feece22b700 1 -- 172.21.15.79:0/2491966552 shutdown_connections 2023-06-23T18:35:46.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.961+0000 7feece22b700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec806a3d0 0x7feec815a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:46.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.961+0000 7feece22b700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7feea806ac50 0x7feea806d100 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:46.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.961+0000 7feece22b700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feec8149af0 0x7feec80cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:46.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.961+0000 7feece22b700 1 --2- 172.21.15.79:0/2491966552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feec8154010 0x7feec8069e90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:46.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.961+0000 7feece22b700 1 -- 172.21.15.79:0/2491966552 >> 172.21.15.79:0/2491966552 conn(0x7feec806afa0 msgr2=0x7feec8154670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:46.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.961+0000 7feece22b700 1 -- 172.21.15.79:0/2491966552 shutdown_connections 2023-06-23T18:35:46.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:46.961+0000 7feece22b700 1 -- 172.21.15.79:0/2491966552 wait complete. 2023-06-23T18:35:46.968 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'rbd' created 2023-06-23T18:35:46.978 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-06-23T18:35:47.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.095+0000 7f627ed35700 1 Processor -- start 2023-06-23T18:35:47.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.095+0000 7f627ed35700 1 -- start start 2023-06-23T18:35:47.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f627ed35700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f627815a2d0 0x7f627806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:47.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f627ed35700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f62780aee00 0x7f6278071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:47.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f627ed35700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627815a6a0 con 0x7f6278072140 2023-06-23T18:35:47.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f627ed35700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62780cc170 con 0x7f627815a2d0 2023-06-23T18:35:47.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f627ed35700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62780cb910 con 0x7f62780aee00 2023-06-23T18:35:47.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f627cad1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f627815a2d0 0x7f627806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:47.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f627d2d2700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f6278072140 0x7f627815e120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41818/0 (socket says 172.21.15.79:41818) 2023-06-23T18:35:47.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f627d2d2700 1 -- 172.21.15.79:0/4132826791 learned_addr learned my addr 172.21.15.79:0/4132826791 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:47.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f6277fff700 1 --2- 172.21.15.79:0/4132826791 >> v2:172.21.15.155:3300/0 conn(0x7f62780aee00 0x7f6278071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:47.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f627cad1700 1 -- 172.21.15.79:0/4132826791 >> v2:172.21.15.155:3300/0 conn(0x7f62780aee00 msgr2=0x7f6278071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:47.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f627cad1700 1 --2- 172.21.15.79:0/4132826791 >> v2:172.21.15.155:3300/0 conn(0x7f62780aee00 0x7f6278071490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:47.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f627cad1700 1 -- 172.21.15.79:0/4132826791 >> v1:172.21.15.79:6789/0 conn(0x7f6278072140 legacy=0x7f627815e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:47.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f627cad1700 1 -- 172.21.15.79:0/4132826791 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62780ca200 con 0x7f627815a2d0 2023-06-23T18:35:47.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f62777fe700 1 -- 172.21.15.79:0/4132826791 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199509772 0 0) 0x7f6270001490 con 0x7f6278072140 2023-06-23T18:35:47.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f6277fff700 1 --2- 172.21.15.79:0/4132826791 >> v2:172.21.15.155:3300/0 conn(0x7f62780aee00 0x7f6278071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:35:47.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f627cad1700 1 --2- 172.21.15.79:0/4132826791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f627815a2d0 0x7f627806eeb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6278070040 tx=0x7f626c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17031dfabf3e74d0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:47.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.096+0000 7f62777fe700 1 -- 172.21.15.79:0/4132826791 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c011830 con 0x7f627815a2d0 2023-06-23T18:35:47.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.097+0000 7f62777fe700 1 -- 172.21.15.79:0/4132826791 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626c011990 con 0x7f627815a2d0 2023-06-23T18:35:47.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.097+0000 7f62777fe700 1 -- 172.21.15.79:0/4132826791 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c011d20 con 0x7f627815a2d0 2023-06-23T18:35:47.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.097+0000 7f627ed35700 1 -- 172.21.15.79:0/4132826791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f627815a2d0 msgr2=0x7f627806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:47.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.097+0000 7f627ed35700 1 --2- 172.21.15.79:0/4132826791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f627815a2d0 0x7f627806eeb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6278070040 tx=0x7f626c009b20 comp rx=0 tx=0).stop 2023-06-23T18:35:47.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.097+0000 7f627ed35700 1 -- 172.21.15.79:0/4132826791 shutdown_connections 2023-06-23T18:35:47.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.097+0000 7f627ed35700 1 --2- 172.21.15.79:0/4132826791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f627815a2d0 0x7f627806eeb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:47.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.097+0000 7f627ed35700 1 --2- 172.21.15.79:0/4132826791 >> v2:172.21.15.155:3300/0 conn(0x7f62780aee00 0x7f6278071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:47.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.097+0000 7f627ed35700 1 -- 172.21.15.79:0/4132826791 >> 172.21.15.79:0/4132826791 conn(0x7f627806e200 msgr2=0x7f62780cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:47.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.097+0000 7f627ed35700 1 -- 172.21.15.79:0/4132826791 shutdown_connections 2023-06-23T18:35:47.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.097+0000 7f627ed35700 1 -- 172.21.15.79:0/4132826791 wait complete. 2023-06-23T18:35:47.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.098+0000 7f627ed35700 1 Processor -- start 2023-06-23T18:35:47.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.098+0000 7f627ed35700 1 -- start start 2023-06-23T18:35:47.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.099+0000 7f627ed35700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6278072140 0x7f62780c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:47.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.099+0000 7f627ed35700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62780aee00 0x7f6278151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:47.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.099+0000 7f627ed35700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f627815a2d0 0x7f6278151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:47.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.099+0000 7f627ed35700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6278151cb0 con 0x7f6278072140 2023-06-23T18:35:47.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.099+0000 7f627ed35700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62780c36a0 con 0x7f627815a2d0 2023-06-23T18:35:47.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.099+0000 7f627ed35700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627815ba80 con 0x7f62780aee00 2023-06-23T18:35:47.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.099+0000 7f627cad1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6278072140 0x7f62780c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:47.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.099+0000 7f627cad1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6278072140 0x7f62780c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38780/0 (socket says 172.21.15.79:38780) 2023-06-23T18:35:47.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.099+0000 7f627cad1700 1 -- 172.21.15.79:0/3700123530 learned_addr learned my addr 172.21.15.79:0/3700123530 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:47.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.099+0000 7f627cad1700 1 -- 172.21.15.79:0/3700123530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62780aee00 msgr2=0x7f6278151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:47.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.099+0000 7f627cad1700 1 --2- 172.21.15.79:0/3700123530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62780aee00 0x7f6278151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:47.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.099+0000 7f627cad1700 1 -- 172.21.15.79:0/3700123530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f627815a2d0 msgr2=0x7f6278151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:47.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.099+0000 7f627cad1700 1 --2- 172.21.15.79:0/3700123530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f627815a2d0 0x7f6278151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:47.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.099+0000 7f627cad1700 1 -- 172.21.15.79:0/3700123530 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c0094a0 con 0x7f6278072140 2023-06-23T18:35:47.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.100+0000 7f627cad1700 1 --2- 172.21.15.79:0/3700123530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6278072140 0x7f62780c5de0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f626c014040 tx=0x7f626c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:47.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.100+0000 7f62757fa700 1 -- 172.21.15.79:0/3700123530 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c0038a0 con 0x7f6278072140 2023-06-23T18:35:47.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.100+0000 7f62757fa700 1 -- 172.21.15.79:0/3700123530 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626c01e070 con 0x7f6278072140 2023-06-23T18:35:47.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.100+0000 7f627ed35700 1 -- 172.21.15.79:0/3700123530 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627815bd60 con 0x7f6278072140 2023-06-23T18:35:47.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.100+0000 7f62757fa700 1 -- 172.21.15.79:0/3700123530 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c003c40 con 0x7f6278072140 2023-06-23T18:35:47.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.100+0000 7f627ed35700 1 -- 172.21.15.79:0/3700123530 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627815c320 con 0x7f6278072140 2023-06-23T18:35:47.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.101+0000 7f627ed35700 1 -- 172.21.15.79:0/3700123530 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627806a190 con 0x7f6278072140 2023-06-23T18:35:47.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.101+0000 7f62757fa700 1 -- 172.21.15.79:0/3700123530 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f626c02d450 con 0x7f6278072140 2023-06-23T18:35:47.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.101+0000 7f62757fa700 1 --2- 172.21.15.79:0/3700123530 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f625c06ac50 0x7f625c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:47.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.102+0000 7f62757fa700 1 -- 172.21.15.79:0/3700123530 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 5188+0+0 (secure 0 0 0) 0x7f626c08a140 con 0x7f6278072140 2023-06-23T18:35:47.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.102+0000 7f6277fff700 1 --2- 172.21.15.79:0/3700123530 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f625c06ac50 0x7f625c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:47.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.102+0000 7f6277fff700 1 --2- 172.21.15.79:0/3700123530 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f625c06ac50 0x7f625c06d100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62781f4430 tx=0x7f6264008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:47.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.106+0000 7f62757fa700 1 -- 172.21.15.79:0/3700123530 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f626c0556c0 con 0x7f6278072140 2023-06-23T18:35:47.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.297+0000 7f627ed35700 1 -- 172.21.15.79:0/3700123530 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f62780672c0 con 0x7f6278072140 2023-06-23T18:35:47.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.960+0000 7f62757fa700 1 -- 172.21.15.79:0/3700123530 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f626c058ce0 con 0x7f6278072140 2023-06-23T18:35:47.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.964+0000 7f627ed35700 1 -- 172.21.15.79:0/3700123530 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f625c06ac50 msgr2=0x7f625c06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:47.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.964+0000 7f627ed35700 1 --2- 172.21.15.79:0/3700123530 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f625c06ac50 0x7f625c06d100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62781f4430 tx=0x7f6264008040 comp rx=0 tx=0).stop 2023-06-23T18:35:47.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.964+0000 7f627ed35700 1 -- 172.21.15.79:0/3700123530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6278072140 msgr2=0x7f62780c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:47.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.964+0000 7f627ed35700 1 --2- 172.21.15.79:0/3700123530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6278072140 0x7f62780c5de0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f626c014040 tx=0x7f626c004750 comp rx=0 tx=0).stop 2023-06-23T18:35:47.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.964+0000 7f627ed35700 1 -- 172.21.15.79:0/3700123530 shutdown_connections 2023-06-23T18:35:47.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.964+0000 7f627ed35700 1 --2- 172.21.15.79:0/3700123530 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f625c06ac50 0x7f625c06d100 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:47.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.964+0000 7f627ed35700 1 --2- 172.21.15.79:0/3700123530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f627815a2d0 0x7f6278151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:47.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.964+0000 7f627ed35700 1 --2- 172.21.15.79:0/3700123530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6278072140 0x7f62780c5de0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:47.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.964+0000 7f627ed35700 1 --2- 172.21.15.79:0/3700123530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62780aee00 0x7f6278151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:47.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.964+0000 7f627ed35700 1 -- 172.21.15.79:0/3700123530 >> 172.21.15.79:0/3700123530 conn(0x7f627806e200 msgr2=0x7f627806f810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:47.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.964+0000 7f627ed35700 1 -- 172.21.15.79:0/3700123530 shutdown_connections 2023-06-23T18:35:47.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:47.964+0000 7f627ed35700 1 -- 172.21.15.79:0/3700123530 wait complete. 2023-06-23T18:35:47.971 INFO:teuthology.orchestra.run.smithi079.stderr:enabled application 'rbd' on pool 'rbd' 2023-06-23T18:35:47.984 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-06-23T18:35:47.985 INFO:tasks.ceph.mds.a:Restarting daemon 2023-06-23T18:35:47.985 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-06-23T18:35:47.987 INFO:tasks.ceph.mds.a:Started 2023-06-23T18:35:47.988 INFO:tasks.ceph.mds.c:Restarting daemon 2023-06-23T18:35:47.988 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-06-23T18:35:47.990 INFO:tasks.ceph.mds.c:Started 2023-06-23T18:35:47.990 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-23T18:35:47.990 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-23T18:35:47.993 INFO:tasks.ceph.mds.b:Started 2023-06-23T18:35:47.993 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-23T18:35:47.993 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-23T18:35:47.995 INFO:tasks.ceph.mds.d:Started 2023-06-23T18:35:47.996 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-06-23T18:35:47.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:35:48.083 INFO:tasks.ceph.mds.c.smithi079.stdout:starting mds.c at 2023-06-23T18:35:48.089 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-06-23T18:35:48.090 INFO:tasks.ceph.mds.a.smithi079.stdout:starting mds.a at 2023-06-23T18:35:48.090 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-06-23T18:35:48.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.137+0000 7fa0ddbfe700 1 Processor -- start 2023-06-23T18:35:48.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.137+0000 7fa0ddbfe700 1 -- start start 2023-06-23T18:35:48.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.138+0000 7fa0ddbfe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa0d80cdad0 0x7fa0d80ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.138+0000 7fa0ddbfe700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d815cd50 con 0x7fa0d8159170 2023-06-23T18:35:48.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.138+0000 7fa0ddbfe700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d80ceaa0 con 0x7fa0d80ce620 2023-06-23T18:35:48.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.138+0000 7fa0ddbfe700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d815cfa0 con 0x7fa0d80cdad0 2023-06-23T18:35:48.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.138+0000 7fa0d77fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa0d8159170 0x7fa0d80cd3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41886/0 (socket says 172.21.15.79:41886) 2023-06-23T18:35:48.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.138+0000 7fa0d77fe700 1 -- 172.21.15.79:0/2148673705 learned_addr learned my addr 172.21.15.79:0/2148673705 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:48.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.138+0000 7fa0d6ffd700 1 --2- 172.21.15.79:0/2148673705 >> v2:172.21.15.155:3300/0 conn(0x7fa0d80cdad0 0x7fa0d80ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.138+0000 7fa0d67fc700 1 -- 172.21.15.79:0/2148673705 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866725108 0 0) 0x7fa0d815cd50 con 0x7fa0d8159170 2023-06-23T18:35:48.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.138+0000 7fa0d67fc700 1 -- 172.21.15.79:0/2148673705 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0c00035b0 con 0x7fa0d8159170 2023-06-23T18:35:48.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d67fc700 1 -- 172.21.15.79:0/2148673705 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4187472997 0 0) 0x7fa0c00035b0 con 0x7fa0d8159170 2023-06-23T18:35:48.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d67fc700 1 -- 172.21.15.79:0/2148673705 >> v2:172.21.15.155:3300/0 conn(0x7fa0d80cdad0 msgr2=0x7fa0d80ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:48.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d67fc700 1 --2- 172.21.15.79:0/2148673705 >> v2:172.21.15.155:3300/0 conn(0x7fa0d80cdad0 0x7fa0d80ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d67fc700 1 -- 172.21.15.79:0/2148673705 >> v1:172.21.15.155:6789/0 conn(0x7fa0d80ce620 legacy=0x7fa0d80cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d67fc700 1 -- 172.21.15.79:0/2148673705 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d815d280 con 0x7fa0d8159170 2023-06-23T18:35:48.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d67fc700 1 -- 172.21.15.79:0/2148673705 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fa0c8001670 con 0x7fa0d8159170 2023-06-23T18:35:48.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d67fc700 1 -- 172.21.15.79:0/2148673705 >> v1:172.21.15.79:6789/0 conn(0x7fa0d8159170 legacy=0x7fa0d80cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d67fc700 1 --2- 172.21.15.79:0/2148673705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c0003a00 0x7fa0c0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d67fc700 1 --2- 172.21.15.79:0/2148673705 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0c0004850 0x7fa0c0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d67fc700 1 --2- 172.21.15.79:0/2148673705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0c0005200 0x7fa0c000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d67fc700 1 -- 172.21.15.79:0/2148673705 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d815cfa0 con 0x7fa0c0005200 2023-06-23T18:35:48.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d67fc700 1 -- 172.21.15.79:0/2148673705 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c000c110 con 0x7fa0c0003a00 2023-06-23T18:35:48.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d67fc700 1 -- 172.21.15.79:0/2148673705 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c000c2d0 con 0x7fa0c0004850 2023-06-23T18:35:48.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d7fff700 1 --2- 172.21.15.79:0/2148673705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0c0005200 0x7fa0c000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d77fe700 1 --2- 172.21.15.79:0/2148673705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c0003a00 0x7fa0c0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d6ffd700 1 --2- 172.21.15.79:0/2148673705 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0c0004850 0x7fa0c0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d7fff700 1 -- 172.21.15.79:0/2148673705 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0c0004850 msgr2=0x7fa0c0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:48.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d7fff700 1 --2- 172.21.15.79:0/2148673705 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0c0004850 0x7fa0c0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d7fff700 1 -- 172.21.15.79:0/2148673705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c0003a00 msgr2=0x7fa0c0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:48.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d7fff700 1 --2- 172.21.15.79:0/2148673705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c0003a00 0x7fa0c0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d7fff700 1 -- 172.21.15.79:0/2148673705 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c00035b0 con 0x7fa0c0005200 2023-06-23T18:35:48.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d77fe700 1 --2- 172.21.15.79:0/2148673705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c0003a00 0x7fa0c0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:48.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d7fff700 1 -- 172.21.15.79:0/2148673705 reap_dead start 2023-06-23T18:35:48.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.139+0000 7fa0d7fff700 1 --2- 172.21.15.79:0/2148673705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0c0005200 0x7fa0c000ba10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000c190 tx=0x7fa0c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35096b5226a4b567 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:48.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.140+0000 7fa0d67fc700 1 -- 172.21.15.79:0/2148673705 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0001db0 con 0x7fa0c0005200 2023-06-23T18:35:48.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.140+0000 7fa0d67fc700 1 -- 172.21.15.79:0/2148673705 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d000ccf0 con 0x7fa0c0005200 2023-06-23T18:35:48.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.140+0000 7fa0d67fc700 1 -- 172.21.15.79:0/2148673705 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0003c40 con 0x7fa0c0005200 2023-06-23T18:35:48.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.140+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/2148673705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0c0005200 msgr2=0x7fa0c000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.140+0000 7fa0ddbfe700 1 --2- 172.21.15.79:0/2148673705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0c0005200 0x7fa0c000ba10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000c190 tx=0x7fa0c0005880 comp rx=0 tx=0).stop 2023-06-23T18:35:48.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.140+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/2148673705 shutdown_connections 2023-06-23T18:35:48.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.140+0000 7fa0ddbfe700 1 --2- 172.21.15.79:0/2148673705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0c0005200 0x7fa0c000ba10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.140+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/2148673705 >> 172.21.15.79:0/2148673705 conn(0x7fa0d806d660 msgr2=0x7fa0d806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:48.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.141+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/2148673705 shutdown_connections 2023-06-23T18:35:48.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.141+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/2148673705 wait complete. 2023-06-23T18:35:48.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.142+0000 7fa0ddbfe700 1 Processor -- start 2023-06-23T18:35:48.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.142+0000 7fa0ddbfe700 1 -- start start 2023-06-23T18:35:48.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0ddbfe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0c0005200 0x7fa0d815b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0ddbfe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0d8069e90 0x7fa0d806a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0ddbfe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0d81f49d0 0x7fa0d81f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0ddbfe700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d806a840 con 0x7fa0d8069e90 2023-06-23T18:35:48.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0ddbfe700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d815baf0 con 0x7fa0d81f49d0 2023-06-23T18:35:48.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0ddbfe700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d80b03e0 con 0x7fa0c0005200 2023-06-23T18:35:48.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0d6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0d8069e90 0x7fa0d806a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0d6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0d8069e90 0x7fa0d806a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38858/0 (socket says 172.21.15.79:38858) 2023-06-23T18:35:48.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0d6ffd700 1 -- 172.21.15.79:0/1866772153 learned_addr learned my addr 172.21.15.79:0/1866772153 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:48.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0d7fff700 1 --2- 172.21.15.79:0/1866772153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0d81f49d0 0x7fa0d81f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0d77fe700 1 --2- 172.21.15.79:0/1866772153 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0c0005200 0x7fa0d815b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0d6ffd700 1 -- 172.21.15.79:0/1866772153 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0c0005200 msgr2=0x7fa0d815b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0d6ffd700 1 --2- 172.21.15.79:0/1866772153 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0c0005200 0x7fa0d815b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0d6ffd700 1 -- 172.21.15.79:0/1866772153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0d81f49d0 msgr2=0x7fa0d81f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0d6ffd700 1 --2- 172.21.15.79:0/1866772153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0d81f49d0 0x7fa0d81f4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0d6ffd700 1 -- 172.21.15.79:0/1866772153 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d81f5380 con 0x7fa0d8069e90 2023-06-23T18:35:48.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0d6ffd700 1 --2- 172.21.15.79:0/1866772153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0d8069e90 0x7fa0d806a300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc011990 tx=0x7fa0cc011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:48.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0bffff700 1 -- 172.21.15.79:0/1866772153 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00f6d0 con 0x7fa0d8069e90 2023-06-23T18:35:48.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0bffff700 1 -- 172.21.15.79:0/1866772153 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc00f830 con 0x7fa0d8069e90 2023-06-23T18:35:48.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0bffff700 1 -- 172.21.15.79:0/1866772153 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00fbe0 con 0x7fa0d8069e90 2023-06-23T18:35:48.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.143+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/1866772153 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d815bd20 con 0x7fa0d8069e90 2023-06-23T18:35:48.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.144+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/1866772153 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d815c2e0 con 0x7fa0d8069e90 2023-06-23T18:35:48.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.145+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/1866772153 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d80b0520 con 0x7fa0d8069e90 2023-06-23T18:35:48.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.145+0000 7fa0bffff700 1 -- 172.21.15.79:0/1866772153 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fa0cc014440 con 0x7fa0d8069e90 2023-06-23T18:35:48.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.145+0000 7fa0bffff700 1 --2- 172.21.15.79:0/1866772153 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa0c406ac50 0x7fa0c406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.146+0000 7fa0bffff700 1 -- 172.21.15.79:0/1866772153 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa0cc0896e0 con 0x7fa0d8069e90 2023-06-23T18:35:48.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.146+0000 7fa0d77fe700 1 --2- 172.21.15.79:0/1866772153 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa0c406ac50 0x7fa0c406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.146+0000 7fa0d77fe700 1 --2- 172.21.15.79:0/1866772153 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa0c406ac50 0x7fa0c406d100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8001200 tx=0x7fa0c8001110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:48.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.150+0000 7fa0bffff700 1 -- 172.21.15.79:0/1866772153 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa0cc054c70 con 0x7fa0d8069e90 2023-06-23T18:35:48.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.359+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/1866772153 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0d80b5770 con 0x7fa0d8069e90 2023-06-23T18:35:48.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.360+0000 7fa0bffff700 1 -- 172.21.15.79:0/1866772153 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+14617 (secure 0 0 0) 0x7fa0cc058290 con 0x7fa0d8069e90 2023-06-23T18:35:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:48.365 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":15,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:47.954609+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:35:48.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.362+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/1866772153 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa0c406ac50 msgr2=0x7fa0c406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.362+0000 7fa0ddbfe700 1 --2- 172.21.15.79:0/1866772153 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa0c406ac50 0x7fa0c406d100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8001200 tx=0x7fa0c8001110 comp rx=0 tx=0).stop 2023-06-23T18:35:48.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.362+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/1866772153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0d8069e90 msgr2=0x7fa0d806a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.362+0000 7fa0ddbfe700 1 --2- 172.21.15.79:0/1866772153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0d8069e90 0x7fa0d806a300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc011990 tx=0x7fa0cc011ca0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.362+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/1866772153 shutdown_connections 2023-06-23T18:35:48.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.362+0000 7fa0ddbfe700 1 --2- 172.21.15.79:0/1866772153 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa0c406ac50 0x7fa0c406d100 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.362+0000 7fa0ddbfe700 1 --2- 172.21.15.79:0/1866772153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0d81f49d0 0x7fa0d81f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.362+0000 7fa0ddbfe700 1 --2- 172.21.15.79:0/1866772153 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0c0005200 0x7fa0d815b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.362+0000 7fa0ddbfe700 1 --2- 172.21.15.79:0/1866772153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0d8069e90 0x7fa0d806a300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.362+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/1866772153 >> 172.21.15.79:0/1866772153 conn(0x7fa0d806d660 msgr2=0x7fa0d8158270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:48.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.362+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/1866772153 shutdown_connections 2023-06-23T18:35:48.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.362+0000 7fa0ddbfe700 1 -- 172.21.15.79:0/1866772153 wait complete. 2023-06-23T18:35:48.379 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:35:48.380 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:35:48.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.498+0000 7fafa1be0700 1 Processor -- start 2023-06-23T18:35:48.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.498+0000 7fafa1be0700 1 -- start start 2023-06-23T18:35:48.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7fafa1be0700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7faf9c156ae0 0x7faf9c15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7fafa1be0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7faf9c0cc060 0x7faf9c0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7fafa1be0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf9c071710 0x7faf9c071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7fafa1be0700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c0cc9f0 con 0x7faf9c0cc060 2023-06-23T18:35:48.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7fafa1be0700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c15daf0 con 0x7faf9c156ae0 2023-06-23T18:35:48.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7fafa1be0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c15a560 con 0x7faf9c071710 2023-06-23T18:35:48.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7faf9affd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7faf9c0cc060 0x7faf9c0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7faf9affd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7faf9c0cc060 0x7faf9c0cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38872/0 (socket says 172.21.15.79:38872) 2023-06-23T18:35:48.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7faf9affd700 1 -- 172.21.15.79:0/2003217155 learned_addr learned my addr 172.21.15.79:0/2003217155 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:48.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7faf9b7fe700 1 --2- 172.21.15.79:0/2003217155 >> v2:172.21.15.155:3300/0 conn(0x7faf9c156ae0 0x7faf9c15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7faf9bfff700 1 --2- 172.21.15.79:0/2003217155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf9c071710 0x7faf9c071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7faf9affd700 1 -- 172.21.15.79:0/2003217155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf9c071710 msgr2=0x7faf9c071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:48.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7faf9affd700 1 --2- 172.21.15.79:0/2003217155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf9c071710 0x7faf9c071b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7faf9affd700 1 -- 172.21.15.79:0/2003217155 >> v2:172.21.15.155:3300/0 conn(0x7faf9c156ae0 msgr2=0x7faf9c15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:48.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7faf9affd700 1 --2- 172.21.15.79:0/2003217155 >> v2:172.21.15.155:3300/0 conn(0x7faf9c156ae0 0x7faf9c15d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7faf9affd700 1 -- 172.21.15.79:0/2003217155 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c159950 con 0x7faf9c0cc060 2023-06-23T18:35:48.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7faf9bfff700 1 --2- 172.21.15.79:0/2003217155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf9c071710 0x7faf9c071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:35:48.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.499+0000 7faf9b7fe700 1 --2- 172.21.15.79:0/2003217155 >> v2:172.21.15.155:3300/0 conn(0x7faf9c156ae0 0x7faf9c15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:35:48.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.500+0000 7faf9affd700 1 --2- 172.21.15.79:0/2003217155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9c0cc060 0x7faf9c0cc4b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0d0910 tx=0x7faf8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30b8a3b6915d0ab server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:48.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.500+0000 7faf9a7fc700 1 -- 172.21.15.79:0/2003217155 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00b870 con 0x7faf9c0cc060 2023-06-23T18:35:48.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.500+0000 7faf9a7fc700 1 -- 172.21.15.79:0/2003217155 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c00b9d0 con 0x7faf9c0cc060 2023-06-23T18:35:48.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.500+0000 7faf9a7fc700 1 -- 172.21.15.79:0/2003217155 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00bd60 con 0x7faf9c0cc060 2023-06-23T18:35:48.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.500+0000 7fafa1be0700 1 -- 172.21.15.79:0/2003217155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9c0cc060 msgr2=0x7faf9c0cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.500+0000 7fafa1be0700 1 --2- 172.21.15.79:0/2003217155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9c0cc060 0x7faf9c0cc4b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0d0910 tx=0x7faf8c009bd0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.500+0000 7fafa1be0700 1 -- 172.21.15.79:0/2003217155 shutdown_connections 2023-06-23T18:35:48.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.500+0000 7fafa1be0700 1 --2- 172.21.15.79:0/2003217155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf9c071710 0x7faf9c071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.500+0000 7fafa1be0700 1 --2- 172.21.15.79:0/2003217155 >> v2:172.21.15.155:3300/0 conn(0x7faf9c156ae0 0x7faf9c15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.500+0000 7fafa1be0700 1 --2- 172.21.15.79:0/2003217155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9c0cc060 0x7faf9c0cc4b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.500+0000 7fafa1be0700 1 -- 172.21.15.79:0/2003217155 >> 172.21.15.79:0/2003217155 conn(0x7faf9c06afa0 msgr2=0x7faf9c06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:48.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.500+0000 7fafa1be0700 1 -- 172.21.15.79:0/2003217155 shutdown_connections 2023-06-23T18:35:48.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.500+0000 7fafa1be0700 1 -- 172.21.15.79:0/2003217155 wait complete. 2023-06-23T18:35:48.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.502+0000 7fafa1be0700 1 Processor -- start 2023-06-23T18:35:48.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.502+0000 7fafa1be0700 1 -- start start 2023-06-23T18:35:48.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.502+0000 7fafa1be0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf9c071710 0x7faf9c1f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.502+0000 7fafa1be0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9c0cc060 0x7faf9c0cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.502+0000 7fafa1be0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf9c156ae0 0x7faf9c14d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.502+0000 7fafa1be0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c1f22e0 con 0x7faf9c0cc060 2023-06-23T18:35:48.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.502+0000 7fafa1be0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c1f34e0 con 0x7faf9c071710 2023-06-23T18:35:48.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.502+0000 7fafa1be0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0ae750 con 0x7faf9c156ae0 2023-06-23T18:35:48.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7faf9affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9c0cc060 0x7faf9c0cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7faf9affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9c0cc060 0x7faf9c0cff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38878/0 (socket says 172.21.15.79:38878) 2023-06-23T18:35:48.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7faf9affd700 1 -- 172.21.15.79:0/2736623446 learned_addr learned my addr 172.21.15.79:0/2736623446 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:48.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7faf9bfff700 1 --2- 172.21.15.79:0/2736623446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf9c156ae0 0x7faf9c14d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7faf9b7fe700 1 --2- 172.21.15.79:0/2736623446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf9c071710 0x7faf9c1f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7faf9affd700 1 -- 172.21.15.79:0/2736623446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf9c156ae0 msgr2=0x7faf9c14d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7faf9affd700 1 --2- 172.21.15.79:0/2736623446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf9c156ae0 0x7faf9c14d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7faf9affd700 1 -- 172.21.15.79:0/2736623446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf9c071710 msgr2=0x7faf9c1f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7faf9affd700 1 --2- 172.21.15.79:0/2736623446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf9c071710 0x7faf9c1f2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7faf9affd700 1 -- 172.21.15.79:0/2736623446 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c009550 con 0x7faf9c0cc060 2023-06-23T18:35:48.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7faf9affd700 1 --2- 172.21.15.79:0/2736623446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9c0cc060 0x7faf9c0cff00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0d0970 tx=0x7faf8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:48.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7faf93fff700 1 -- 172.21.15.79:0/2736623446 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c004480 con 0x7faf9c0cc060 2023-06-23T18:35:48.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7faf93fff700 1 -- 172.21.15.79:0/2736623446 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c0045e0 con 0x7faf9c0cc060 2023-06-23T18:35:48.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7faf93fff700 1 -- 172.21.15.79:0/2736623446 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c0038e0 con 0x7faf9c0cc060 2023-06-23T18:35:48.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7fafa1be0700 1 -- 172.21.15.79:0/2736623446 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c1f2b10 con 0x7faf9c0cc060 2023-06-23T18:35:48.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.503+0000 7fafa1be0700 1 -- 172.21.15.79:0/2736623446 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c1f24c0 con 0x7faf9c0cc060 2023-06-23T18:35:48.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.504+0000 7fafa1be0700 1 -- 172.21.15.79:0/2736623446 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9c0672c0 con 0x7faf9c0cc060 2023-06-23T18:35:48.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.505+0000 7faf93fff700 1 -- 172.21.15.79:0/2736623446 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7faf8c021070 con 0x7faf9c0cc060 2023-06-23T18:35:48.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.505+0000 7faf93fff700 1 --2- 172.21.15.79:0/2736623446 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7faf7c06ac50 0x7faf7c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.505+0000 7faf93fff700 1 -- 172.21.15.79:0/2736623446 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7faf8c08a630 con 0x7faf9c0cc060 2023-06-23T18:35:48.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.509+0000 7faf93fff700 1 -- 172.21.15.79:0/2736623446 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faf8c055cc0 con 0x7faf9c0cc060 2023-06-23T18:35:48.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.514+0000 7faf9b7fe700 1 --2- 172.21.15.79:0/2736623446 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7faf7c06ac50 0x7faf7c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.514+0000 7faf9b7fe700 1 --2- 172.21.15.79:0/2736623446 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7faf7c06ac50 0x7faf7c06d100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faf84006f10 tx=0x7faf84008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:48.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.702+0000 7fafa1be0700 1 -- 172.21.15.79:0/2736623446 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faf9c0b1be0 con 0x7faf9c0cc060 2023-06-23T18:35:48.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.702+0000 7faf93fff700 1 -- 172.21.15.79:0/2736623446 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7faf8c08a030 con 0x7faf9c0cc060 2023-06-23T18:35:48.707 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:35:48.709 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.704+0000 7fafa1be0700 1 -- 172.21.15.79:0/2736623446 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7faf7c06ac50 msgr2=0x7faf7c06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.709 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.704+0000 7fafa1be0700 1 --2- 172.21.15.79:0/2736623446 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7faf7c06ac50 0x7faf7c06d100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faf84006f10 tx=0x7faf84008040 comp rx=0 tx=0).stop 2023-06-23T18:35:48.709 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.704+0000 7fafa1be0700 1 -- 172.21.15.79:0/2736623446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9c0cc060 msgr2=0x7faf9c0cff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.709 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.704+0000 7fafa1be0700 1 --2- 172.21.15.79:0/2736623446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9c0cc060 0x7faf9c0cff00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0d0970 tx=0x7faf8c004750 comp rx=0 tx=0).stop 2023-06-23T18:35:48.709 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.704+0000 7fafa1be0700 1 -- 172.21.15.79:0/2736623446 shutdown_connections 2023-06-23T18:35:48.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.704+0000 7fafa1be0700 1 --2- 172.21.15.79:0/2736623446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf9c156ae0 0x7faf9c14d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.704+0000 7fafa1be0700 1 --2- 172.21.15.79:0/2736623446 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7faf7c06ac50 0x7faf7c06d100 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.704+0000 7fafa1be0700 1 --2- 172.21.15.79:0/2736623446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf9c071710 0x7faf9c1f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.704+0000 7fafa1be0700 1 --2- 172.21.15.79:0/2736623446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9c0cc060 0x7faf9c0cff00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.704+0000 7fafa1be0700 1 -- 172.21.15.79:0/2736623446 >> 172.21.15.79:0/2736623446 conn(0x7faf9c06afa0 msgr2=0x7faf9c1559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:48.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.704+0000 7fafa1be0700 1 -- 172.21.15.79:0/2736623446 shutdown_connections 2023-06-23T18:35:48.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.704+0000 7fafa1be0700 1 -- 172.21.15.79:0/2736623446 wait complete. 2023-06-23T18:35:48.721 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:35:48.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.845+0000 7ff91b0f0700 1 Processor -- start 2023-06-23T18:35:48.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.845+0000 7ff91b0f0700 1 -- start start 2023-06-23T18:35:48.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91b0f0700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff9140cedf0 0x7ff9140b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91b0f0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9141438f0 0x7ff9140b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91b0f0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff9140d08c0 0x7ff9140d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91b0f0700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9140d1280 con 0x7ff9140d08c0 2023-06-23T18:35:48.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91b0f0700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9140b0b10 con 0x7ff9140cedf0 2023-06-23T18:35:48.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91b0f0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9140b0df0 con 0x7ff9141438f0 2023-06-23T18:35:48.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91968d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff9140d08c0 0x7ff9140d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91968d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff9140d08c0 0x7ff9140d0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38898/0 (socket says 172.21.15.79:38898) 2023-06-23T18:35:48.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91968d700 1 -- 172.21.15.79:0/2777554565 learned_addr learned my addr 172.21.15.79:0/2777554565 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:48.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff913fff700 1 --2- 172.21.15.79:0/2777554565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9141438f0 0x7ff9140b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff918e8c700 1 --2- 172.21.15.79:0/2777554565 >> v2:172.21.15.155:3300/0 conn(0x7ff9140cedf0 0x7ff9140b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91968d700 1 -- 172.21.15.79:0/2777554565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9141438f0 msgr2=0x7ff9140b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:48.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91968d700 1 --2- 172.21.15.79:0/2777554565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9141438f0 0x7ff9140b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91968d700 1 -- 172.21.15.79:0/2777554565 >> v2:172.21.15.155:3300/0 conn(0x7ff9140cedf0 msgr2=0x7ff9140b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:48.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91968d700 1 --2- 172.21.15.79:0/2777554565 >> v2:172.21.15.155:3300/0 conn(0x7ff9140cedf0 0x7ff9140b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91968d700 1 -- 172.21.15.79:0/2777554565 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9140b10d0 con 0x7ff9140d08c0 2023-06-23T18:35:48.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff91968d700 1 --2- 172.21.15.79:0/2777554565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9140d08c0 0x7ff9140d0d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff914066900 tx=0x7ff90c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4d52c86bb7468f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:48.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.846+0000 7ff9137fe700 1 -- 172.21.15.79:0/2777554565 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00e070 con 0x7ff9140d08c0 2023-06-23T18:35:48.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.847+0000 7ff9137fe700 1 -- 172.21.15.79:0/2777554565 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c00b910 con 0x7ff9140d08c0 2023-06-23T18:35:48.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.847+0000 7ff9137fe700 1 -- 172.21.15.79:0/2777554565 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00bcc0 con 0x7ff9140d08c0 2023-06-23T18:35:48.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.847+0000 7ff91b0f0700 1 -- 172.21.15.79:0/2777554565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9140d08c0 msgr2=0x7ff9140d0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.847+0000 7ff91b0f0700 1 --2- 172.21.15.79:0/2777554565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9140d08c0 0x7ff9140d0d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff914066900 tx=0x7ff90c00b330 comp rx=0 tx=0).stop 2023-06-23T18:35:48.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.847+0000 7ff91b0f0700 1 -- 172.21.15.79:0/2777554565 shutdown_connections 2023-06-23T18:35:48.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.847+0000 7ff91b0f0700 1 --2- 172.21.15.79:0/2777554565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9140d08c0 0x7ff9140d0d40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.847+0000 7ff91b0f0700 1 --2- 172.21.15.79:0/2777554565 >> v2:172.21.15.155:3300/0 conn(0x7ff9140cedf0 0x7ff9140b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.847+0000 7ff91b0f0700 1 --2- 172.21.15.79:0/2777554565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9141438f0 0x7ff9140b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.847+0000 7ff91b0f0700 1 -- 172.21.15.79:0/2777554565 >> 172.21.15.79:0/2777554565 conn(0x7ff91406a7b0 msgr2=0x7ff914099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:48.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.847+0000 7ff91b0f0700 1 -- 172.21.15.79:0/2777554565 shutdown_connections 2023-06-23T18:35:48.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.847+0000 7ff91b0f0700 1 -- 172.21.15.79:0/2777554565 wait complete. 2023-06-23T18:35:48.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.848+0000 7ff91b0f0700 1 Processor -- start 2023-06-23T18:35:48.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.848+0000 7ff91b0f0700 1 -- start start 2023-06-23T18:35:48.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.849+0000 7ff91b0f0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9140cedf0 0x7ff9141594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.849+0000 7ff91b0f0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9140d08c0 0x7ff9140ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.849+0000 7ff91b0f0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9141438f0 0x7ff9140ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.849+0000 7ff91b0f0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141f1410 con 0x7ff9140d08c0 2023-06-23T18:35:48.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.849+0000 7ff91b0f0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141e48e0 con 0x7ff9140cedf0 2023-06-23T18:35:48.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.849+0000 7ff91b0f0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141f0dc0 con 0x7ff9141438f0 2023-06-23T18:35:48.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.849+0000 7ff913fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9140d08c0 0x7ff9140ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.849+0000 7ff913fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9140d08c0 0x7ff9140ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38902/0 (socket says 172.21.15.79:38902) 2023-06-23T18:35:48.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.849+0000 7ff913fff700 1 -- 172.21.15.79:0/4152903738 learned_addr learned my addr 172.21.15.79:0/4152903738 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:48.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.849+0000 7ff91968d700 1 --2- 172.21.15.79:0/4152903738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9141438f0 0x7ff9140ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.849+0000 7ff918e8c700 1 --2- 172.21.15.79:0/4152903738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9140cedf0 0x7ff9141594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.850+0000 7ff913fff700 1 -- 172.21.15.79:0/4152903738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9141438f0 msgr2=0x7ff9140ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.850+0000 7ff913fff700 1 --2- 172.21.15.79:0/4152903738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9141438f0 0x7ff9140ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.850+0000 7ff913fff700 1 -- 172.21.15.79:0/4152903738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9140cedf0 msgr2=0x7ff9141594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:48.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.850+0000 7ff913fff700 1 --2- 172.21.15.79:0/4152903738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9140cedf0 0x7ff9141594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:48.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.850+0000 7ff913fff700 1 -- 172.21.15.79:0/4152903738 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c009b30 con 0x7ff9140d08c0 2023-06-23T18:35:48.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.850+0000 7ff913fff700 1 --2- 172.21.15.79:0/4152903738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9140d08c0 0x7ff9140ad530 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff90000e9b0 tx=0x7ff90000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:48.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.850+0000 7ff9117fa700 1 -- 172.21.15.79:0/4152903738 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900009710 con 0x7ff9140d08c0 2023-06-23T18:35:48.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.850+0000 7ff9117fa700 1 -- 172.21.15.79:0/4152903738 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90000ee50 con 0x7ff9140d08c0 2023-06-23T18:35:48.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.850+0000 7ff91b0f0700 1 -- 172.21.15.79:0/4152903738 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9141f10a0 con 0x7ff9140d08c0 2023-06-23T18:35:48.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.850+0000 7ff9117fa700 1 -- 172.21.15.79:0/4152903738 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90000ca50 con 0x7ff9140d08c0 2023-06-23T18:35:48.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.850+0000 7ff91b0f0700 1 -- 172.21.15.79:0/4152903738 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9141e4bc0 con 0x7ff9140d08c0 2023-06-23T18:35:48.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.851+0000 7ff91b0f0700 1 -- 172.21.15.79:0/4152903738 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9141f1580 con 0x7ff9140d08c0 2023-06-23T18:35:48.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.851+0000 7ff9117fa700 1 -- 172.21.15.79:0/4152903738 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7ff90000cbb0 con 0x7ff9140d08c0 2023-06-23T18:35:48.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.852+0000 7ff9117fa700 1 --2- 172.21.15.79:0/4152903738 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff8f806ac50 0x7ff8f806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:48.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.852+0000 7ff918e8c700 1 --2- 172.21.15.79:0/4152903738 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff8f806ac50 0x7ff8f806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:48.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.852+0000 7ff9117fa700 1 -- 172.21.15.79:0/4152903738 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff9000890b0 con 0x7ff9140d08c0 2023-06-23T18:35:48.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.852+0000 7ff918e8c700 1 --2- 172.21.15.79:0/4152903738 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff8f806ac50 0x7ff8f806d100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff908006fd0 tx=0x7ff908008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:48.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:48.856+0000 7ff9117fa700 1 -- 172.21.15.79:0/4152903738 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff900054510 con 0x7ff9140d08c0 2023-06-23T18:35:49.055 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.050+0000 7ff91b0f0700 1 -- 172.21.15.79:0/4152903738 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9140672c0 con 0x7ff9140d08c0 2023-06-23T18:35:49.055 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.050+0000 7ff9117fa700 1 -- 172.21.15.79:0/4152903738 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff90000ce60 con 0x7ff9140d08c0 2023-06-23T18:35:49.055 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:35:49.057 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.052+0000 7ff91b0f0700 1 -- 172.21.15.79:0/4152903738 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff8f806ac50 msgr2=0x7ff8f806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.058 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.052+0000 7ff91b0f0700 1 --2- 172.21.15.79:0/4152903738 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff8f806ac50 0x7ff8f806d100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff908006fd0 tx=0x7ff908008040 comp rx=0 tx=0).stop 2023-06-23T18:35:49.058 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.052+0000 7ff91b0f0700 1 -- 172.21.15.79:0/4152903738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9140d08c0 msgr2=0x7ff9140ad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.058 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.052+0000 7ff91b0f0700 1 --2- 172.21.15.79:0/4152903738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9140d08c0 0x7ff9140ad530 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff90000e9b0 tx=0x7ff90000ecc0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.058 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.053+0000 7ff91b0f0700 1 -- 172.21.15.79:0/4152903738 shutdown_connections 2023-06-23T18:35:49.058 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.053+0000 7ff91b0f0700 1 --2- 172.21.15.79:0/4152903738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9141438f0 0x7ff9140ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.058 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.053+0000 7ff91b0f0700 1 --2- 172.21.15.79:0/4152903738 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff8f806ac50 0x7ff8f806d100 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.058 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.053+0000 7ff91b0f0700 1 --2- 172.21.15.79:0/4152903738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9140cedf0 0x7ff9141594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.059 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.053+0000 7ff91b0f0700 1 --2- 172.21.15.79:0/4152903738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9140d08c0 0x7ff9140ad530 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.059 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.053+0000 7ff91b0f0700 1 -- 172.21.15.79:0/4152903738 >> 172.21.15.79:0/4152903738 conn(0x7ff91406a7b0 msgr2=0x7ff91406fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:49.059 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.053+0000 7ff91b0f0700 1 -- 172.21.15.79:0/4152903738 shutdown_connections 2023-06-23T18:35:49.059 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.053+0000 7ff91b0f0700 1 -- 172.21.15.79:0/4152903738 wait complete. 2023-06-23T18:35:49.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:35:49.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.186+0000 7ff4ab521700 1 Processor -- start 2023-06-23T18:35:49.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.187+0000 7ff4ab521700 1 -- start start 2023-06-23T18:35:49.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.187+0000 7ff4ab521700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a40aee30 0x7ff4a415d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff4ab521700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a41608f0 con 0x7ff4a40b1270 2023-06-23T18:35:49.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff4ab521700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a4160e20 con 0x7ff4a4071490 2023-06-23T18:35:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff4ab521700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40cd4f0 con 0x7ff4a40aee30 2023-06-23T18:35:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff4a8abc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff4a40b1270 0x7ff4a415c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41956/0 (socket says 172.21.15.79:41956) 2023-06-23T18:35:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff4a8abc700 1 -- 172.21.15.79:0/275648271 learned_addr learned my addr 172.21.15.79:0/275648271 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff4a9abe700 1 --2- 172.21.15.79:0/275648271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a40aee30 0x7ff4a415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff49bfff700 1 -- 172.21.15.79:0/275648271 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792190854 0 0) 0x7ff4a41608f0 con 0x7ff4a40b1270 2023-06-23T18:35:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff49bfff700 1 -- 172.21.15.79:0/275648271 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4900035b0 con 0x7ff4a40b1270 2023-06-23T18:35:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff49bfff700 1 -- 172.21.15.79:0/275648271 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049182823 0 0) 0x7ff4a4160e20 con 0x7ff4a4071490 2023-06-23T18:35:49.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff49bfff700 1 -- 172.21.15.79:0/275648271 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4a41608f0 con 0x7ff4a4071490 2023-06-23T18:35:49.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff4a9abe700 1 -- 172.21.15.79:0/275648271 >> v1:172.21.15.155:6789/0 conn(0x7ff4a4071490 legacy=0x7ff4a4071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff4a9abe700 1 -- 172.21.15.79:0/275648271 >> v1:172.21.15.79:6789/0 conn(0x7ff4a40b1270 legacy=0x7ff4a415c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff49bfff700 1 -- 172.21.15.79:0/275648271 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1767701394 0 0) 0x7ff4900035b0 con 0x7ff4a40b1270 2023-06-23T18:35:49.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff4a9abe700 1 -- 172.21.15.79:0/275648271 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a4160b40 con 0x7ff4a40aee30 2023-06-23T18:35:49.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff4a9abe700 1 --2- 172.21.15.79:0/275648271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a40aee30 0x7ff4a415d100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009dc0 tx=0x7ff49c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60d49823474f8eaf server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:49.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff49bfff700 1 -- 172.21.15.79:0/275648271 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00e070 con 0x7ff4a40aee30 2023-06-23T18:35:49.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff49bfff700 1 -- 172.21.15.79:0/275648271 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c003a20 con 0x7ff4a40aee30 2023-06-23T18:35:49.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.188+0000 7ff49bfff700 1 -- 172.21.15.79:0/275648271 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0001670 con 0x7ff4a40aee30 2023-06-23T18:35:49.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.189+0000 7ff4ab521700 1 -- 172.21.15.79:0/275648271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a40aee30 msgr2=0x7ff4a415d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.189+0000 7ff4ab521700 1 --2- 172.21.15.79:0/275648271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a40aee30 0x7ff4a415d100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009dc0 tx=0x7ff49c00b330 comp rx=0 tx=0).stop 2023-06-23T18:35:49.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.189+0000 7ff4ab521700 1 -- 172.21.15.79:0/275648271 shutdown_connections 2023-06-23T18:35:49.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.189+0000 7ff4ab521700 1 --2- 172.21.15.79:0/275648271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a40aee30 0x7ff4a415d100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.189+0000 7ff4ab521700 1 -- 172.21.15.79:0/275648271 >> 172.21.15.79:0/275648271 conn(0x7ff4a406d660 msgr2=0x7ff4a406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:49.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.189+0000 7ff4ab521700 1 -- 172.21.15.79:0/275648271 shutdown_connections 2023-06-23T18:35:49.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.189+0000 7ff4ab521700 1 -- 172.21.15.79:0/275648271 wait complete. 2023-06-23T18:35:49.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.190+0000 7ff4ab521700 1 Processor -- start 2023-06-23T18:35:49.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.190+0000 7ff4ab521700 1 -- start start 2023-06-23T18:35:49.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4ab521700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a4071490 0x7ff4a414bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4ab521700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a40aee30 0x7ff4a40c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4ab521700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a40b1270 0x7ff4a40cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4ab521700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40c5af0 con 0x7ff4a4071490 2023-06-23T18:35:49.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4ab521700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40c4ee0 con 0x7ff4a40b1270 2023-06-23T18:35:49.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4ab521700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40c51c0 con 0x7ff4a40aee30 2023-06-23T18:35:49.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4a92bd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a4071490 0x7ff4a414bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4a92bd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a4071490 0x7ff4a414bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38918/0 (socket says 172.21.15.79:38918) 2023-06-23T18:35:49.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4a92bd700 1 -- 172.21.15.79:0/2239791792 learned_addr learned my addr 172.21.15.79:0/2239791792 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:49.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4a8abc700 1 --2- 172.21.15.79:0/2239791792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a40aee30 0x7ff4a40c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4a9abe700 1 --2- 172.21.15.79:0/2239791792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a40b1270 0x7ff4a40cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4a92bd700 1 -- 172.21.15.79:0/2239791792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a40aee30 msgr2=0x7ff4a40c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4a92bd700 1 --2- 172.21.15.79:0/2239791792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a40aee30 0x7ff4a40c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4a92bd700 1 -- 172.21.15.79:0/2239791792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a40b1270 msgr2=0x7ff4a40cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4a92bd700 1 --2- 172.21.15.79:0/2239791792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a40b1270 0x7ff4a40cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.191+0000 7ff4a92bd700 1 -- 172.21.15.79:0/2239791792 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c009a70 con 0x7ff4a4071490 2023-06-23T18:35:49.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.192+0000 7ff4a92bd700 1 --2- 172.21.15.79:0/2239791792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a4071490 0x7ff4a414bdc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff49400c500 tx=0x7ff49400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:49.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.192+0000 7ff499ffb700 1 -- 172.21.15.79:0/2239791792 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4940052d0 con 0x7ff4a4071490 2023-06-23T18:35:49.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.192+0000 7ff499ffb700 1 -- 172.21.15.79:0/2239791792 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff494005450 con 0x7ff4a4071490 2023-06-23T18:35:49.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.192+0000 7ff4ab521700 1 -- 172.21.15.79:0/2239791792 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a40c54a0 con 0x7ff4a4071490 2023-06-23T18:35:49.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.192+0000 7ff499ffb700 1 -- 172.21.15.79:0/2239791792 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4940038c0 con 0x7ff4a4071490 2023-06-23T18:35:49.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.192+0000 7ff4ab521700 1 -- 172.21.15.79:0/2239791792 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a40c4bd0 con 0x7ff4a4071490 2023-06-23T18:35:49.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.193+0000 7ff4ab521700 1 -- 172.21.15.79:0/2239791792 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a4152320 con 0x7ff4a4071490 2023-06-23T18:35:49.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.193+0000 7ff499ffb700 1 -- 172.21.15.79:0/2239791792 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7ff494003a20 con 0x7ff4a4071490 2023-06-23T18:35:49.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.193+0000 7ff499ffb700 1 --2- 172.21.15.79:0/2239791792 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff48806ac50 0x7ff48806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.194+0000 7ff499ffb700 1 -- 172.21.15.79:0/2239791792 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff4940883b0 con 0x7ff4a4071490 2023-06-23T18:35:49.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.194+0000 7ff4a8abc700 1 --2- 172.21.15.79:0/2239791792 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff48806ac50 0x7ff48806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.194+0000 7ff4a8abc700 1 --2- 172.21.15.79:0/2239791792 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff48806ac50 0x7ff48806d100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0000c00 tx=0x7ff4a0001e20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:49.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.198+0000 7ff499ffb700 1 -- 172.21.15.79:0/2239791792 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff494053890 con 0x7ff4a4071490 2023-06-23T18:35:49.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.427+0000 7ff4ab521700 1 -- 172.21.15.79:0/2239791792 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4a40b5770 con 0x7ff4a4071490 2023-06-23T18:35:49.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.428+0000 7ff499ffb700 1 -- 172.21.15.79:0/2239791792 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7ff494017de0 con 0x7ff4a4071490 2023-06-23T18:35:49.433 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:49.433 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/399719238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":399719238},{"type":"v1","addr":"172.21.15.155:6834","nonce":399719238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1846440916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1846440916},{"type":"v1","addr":"172.21.15.155:6835","nonce":1846440916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3310180303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3310180303},{"type":"v1","addr":"172.21.15.79:6835","nonce":3310180303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3900137670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3900137670},{"type":"v1","addr":"172.21.15.79:6837","nonce":3900137670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-06-23T18:35:49.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.430+0000 7ff4ab521700 1 -- 172.21.15.79:0/2239791792 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff48806ac50 msgr2=0x7ff48806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.430+0000 7ff4ab521700 1 --2- 172.21.15.79:0/2239791792 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff48806ac50 0x7ff48806d100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0000c00 tx=0x7ff4a0001e20 comp rx=0 tx=0).stop 2023-06-23T18:35:49.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.430+0000 7ff4ab521700 1 -- 172.21.15.79:0/2239791792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a4071490 msgr2=0x7ff4a414bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.430+0000 7ff4ab521700 1 --2- 172.21.15.79:0/2239791792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a4071490 0x7ff4a414bdc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff49400c500 tx=0x7ff49400c810 comp rx=0 tx=0).stop 2023-06-23T18:35:49.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.430+0000 7ff4ab521700 1 -- 172.21.15.79:0/2239791792 shutdown_connections 2023-06-23T18:35:49.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.430+0000 7ff4ab521700 1 --2- 172.21.15.79:0/2239791792 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff48806ac50 0x7ff48806d100 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.430+0000 7ff4ab521700 1 --2- 172.21.15.79:0/2239791792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a40b1270 0x7ff4a40cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.430+0000 7ff4ab521700 1 --2- 172.21.15.79:0/2239791792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a4071490 0x7ff4a414bdc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.430+0000 7ff4ab521700 1 --2- 172.21.15.79:0/2239791792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a40aee30 0x7ff4a40c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.430+0000 7ff4ab521700 1 -- 172.21.15.79:0/2239791792 >> 172.21.15.79:0/2239791792 conn(0x7ff4a406d660 msgr2=0x7ff4a40b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:49.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.430+0000 7ff4ab521700 1 -- 172.21.15.79:0/2239791792 shutdown_connections 2023-06-23T18:35:49.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.430+0000 7ff4ab521700 1 -- 172.21.15.79:0/2239791792 wait complete. 2023-06-23T18:35:49.437 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 3 2023-06-23T18:35:49.447 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:35:49.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.565+0000 7f06211d6700 1 Processor -- start 2023-06-23T18:35:49.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.565+0000 7f06211d6700 1 -- start start 2023-06-23T18:35:49.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f06211d6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f061c071490 0x7f061c071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f06211d6700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f061c0aee30 0x7f061c071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f06211d6700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061c1608f0 con 0x7f061c0b1270 2023-06-23T18:35:49.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f06211d6700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f061c0722e0 con 0x7f061c0aee30 2023-06-23T18:35:49.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f06211d6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061c160e20 con 0x7f061c071490 2023-06-23T18:35:49.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f061a59c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f061c0b1270 0x7f061c069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41968/0 (socket says 172.21.15.79:41968) 2023-06-23T18:35:49.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f061a59c700 1 -- 172.21.15.79:0/2080461029 learned_addr learned my addr 172.21.15.79:0/2080461029 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:49.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f061ad9d700 1 --2- 172.21.15.79:0/2080461029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f061c071490 0x7f061c071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f061b59e700 1 --2- 172.21.15.79:0/2080461029 >> v2:172.21.15.155:3300/0 conn(0x7f061c0aee30 0x7f061c071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f0619d9b700 1 -- 172.21.15.79:0/2080461029 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799605775 0 0) 0x7f061c1608f0 con 0x7f061c0b1270 2023-06-23T18:35:49.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f0619d9b700 1 -- 172.21.15.79:0/2080461029 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06040035b0 con 0x7f061c0b1270 2023-06-23T18:35:49.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f0619d9b700 1 -- 172.21.15.79:0/2080461029 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1913200613 0 0) 0x7f06040035b0 con 0x7f061c0b1270 2023-06-23T18:35:49.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f061ad9d700 1 -- 172.21.15.79:0/2080461029 >> v2:172.21.15.155:3300/0 conn(0x7f061c0aee30 msgr2=0x7f061c071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:49.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f061ad9d700 1 --2- 172.21.15.79:0/2080461029 >> v2:172.21.15.155:3300/0 conn(0x7f061c0aee30 0x7f061c071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f061ad9d700 1 -- 172.21.15.79:0/2080461029 >> v1:172.21.15.79:6789/0 conn(0x7f061c0b1270 legacy=0x7f061c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f061ad9d700 1 -- 172.21.15.79:0/2080461029 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c0cd4f0 con 0x7f061c071490 2023-06-23T18:35:49.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f061b59e700 1 --2- 172.21.15.79:0/2080461029 >> v2:172.21.15.155:3300/0 conn(0x7f061c0aee30 0x7f061c071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:49.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f061ad9d700 1 --2- 172.21.15.79:0/2080461029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f061c071490 0x7f061c071860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f060c009840 tx=0x7f060c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f38d6eb3c68be7fe server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:49.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.566+0000 7f0619d9b700 1 -- 172.21.15.79:0/2080461029 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610001670 con 0x7f061c071490 2023-06-23T18:35:49.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.567+0000 7f0619d9b700 1 -- 172.21.15.79:0/2080461029 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060c004030 con 0x7f061c071490 2023-06-23T18:35:49.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.567+0000 7f0619d9b700 1 -- 172.21.15.79:0/2080461029 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c0043a0 con 0x7f061c071490 2023-06-23T18:35:49.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.567+0000 7f06211d6700 1 -- 172.21.15.79:0/2080461029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f061c071490 msgr2=0x7f061c071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.567+0000 7f06211d6700 1 --2- 172.21.15.79:0/2080461029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f061c071490 0x7f061c071860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f060c009840 tx=0x7f060c009b50 comp rx=0 tx=0).stop 2023-06-23T18:35:49.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.567+0000 7f06211d6700 1 -- 172.21.15.79:0/2080461029 shutdown_connections 2023-06-23T18:35:49.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.567+0000 7f06211d6700 1 --2- 172.21.15.79:0/2080461029 >> v2:172.21.15.155:3300/0 conn(0x7f061c0aee30 0x7f061c071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.567+0000 7f06211d6700 1 --2- 172.21.15.79:0/2080461029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f061c071490 0x7f061c071860 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.567+0000 7f06211d6700 1 -- 172.21.15.79:0/2080461029 >> 172.21.15.79:0/2080461029 conn(0x7f061c06d660 msgr2=0x7f061c06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:49.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.567+0000 7f06211d6700 1 -- 172.21.15.79:0/2080461029 shutdown_connections 2023-06-23T18:35:49.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.567+0000 7f06211d6700 1 -- 172.21.15.79:0/2080461029 wait complete. 2023-06-23T18:35:49.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.568+0000 7f06211d6700 1 Processor -- start 2023-06-23T18:35:49.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.569+0000 7f06211d6700 1 -- start start 2023-06-23T18:35:49.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.569+0000 7f06211d6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f061c071490 0x7f061c0c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.569+0000 7f06211d6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f061c0aee30 0x7f061c0c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.569+0000 7f06211d6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f061c0b1270 0x7f061c150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.569+0000 7f06211d6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c0c93d0 con 0x7f061c0aee30 2023-06-23T18:35:49.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.569+0000 7f06211d6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c0c80b0 con 0x7f061c071490 2023-06-23T18:35:49.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.569+0000 7f06211d6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061c0c8330 con 0x7f061c0b1270 2023-06-23T18:35:49.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.569+0000 7f061a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f061c0aee30 0x7f061c0c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.569+0000 7f061a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f061c0aee30 0x7f061c0c7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38934/0 (socket says 172.21.15.79:38934) 2023-06-23T18:35:49.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.569+0000 7f061a59c700 1 -- 172.21.15.79:0/3860478822 learned_addr learned my addr 172.21.15.79:0/3860478822 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:49.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.569+0000 7f061b59e700 1 --2- 172.21.15.79:0/3860478822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f061c0b1270 0x7f061c150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.569+0000 7f061ad9d700 1 --2- 172.21.15.79:0/3860478822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f061c071490 0x7f061c0c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.570+0000 7f061a59c700 1 -- 172.21.15.79:0/3860478822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f061c0b1270 msgr2=0x7f061c150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.570+0000 7f061a59c700 1 --2- 172.21.15.79:0/3860478822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f061c0b1270 0x7f061c150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.570+0000 7f061a59c700 1 -- 172.21.15.79:0/3860478822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f061c071490 msgr2=0x7f061c0c4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.570+0000 7f061a59c700 1 --2- 172.21.15.79:0/3860478822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f061c071490 0x7f061c0c4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.570+0000 7f061a59c700 1 -- 172.21.15.79:0/3860478822 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614009a70 con 0x7f061c0aee30 2023-06-23T18:35:49.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.570+0000 7f061a59c700 1 --2- 172.21.15.79:0/3860478822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f061c0aee30 0x7f061c0c7b70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f06100011e0 tx=0x7f061000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:49.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.570+0000 7f06037fe700 1 -- 172.21.15.79:0/3860478822 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610004c80 con 0x7f061c0aee30 2023-06-23T18:35:49.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.570+0000 7f06037fe700 1 -- 172.21.15.79:0/3860478822 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0610004de0 con 0x7f061c0aee30 2023-06-23T18:35:49.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.570+0000 7f06211d6700 1 -- 172.21.15.79:0/3860478822 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060c0094a0 con 0x7f061c0aee30 2023-06-23T18:35:49.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.570+0000 7f06037fe700 1 -- 172.21.15.79:0/3860478822 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06100072f0 con 0x7f061c0aee30 2023-06-23T18:35:49.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.570+0000 7f06211d6700 1 -- 172.21.15.79:0/3860478822 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c0c88f0 con 0x7f061c0aee30 2023-06-23T18:35:49.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.571+0000 7f06211d6700 1 -- 172.21.15.79:0/3860478822 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c0724a0 con 0x7f061c0aee30 2023-06-23T18:35:49.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.571+0000 7f06037fe700 1 -- 172.21.15.79:0/3860478822 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0610007450 con 0x7f061c0aee30 2023-06-23T18:35:49.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.572+0000 7f06037fe700 1 --2- 172.21.15.79:0/3860478822 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f060806ace0 0x7f060806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.572+0000 7f061ad9d700 1 --2- 172.21.15.79:0/3860478822 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f060806ace0 0x7f060806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.572+0000 7f06037fe700 1 -- 172.21.15.79:0/3860478822 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0610088ef0 con 0x7f061c0aee30 2023-06-23T18:35:49.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.572+0000 7f061ad9d700 1 --2- 172.21.15.79:0/3860478822 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f060806ace0 0x7f060806d190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f060c000c00 tx=0x7f060c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:49.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.576+0000 7f06037fe700 1 -- 172.21.15.79:0/3860478822 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0610054250 con 0x7f061c0aee30 2023-06-23T18:35:49.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.763+0000 7f06211d6700 1 -- 172.21.15.79:0/3860478822 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f061c0b5770 con 0x7f061c0aee30 2023-06-23T18:35:49.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.764+0000 7f06037fe700 1 -- 172.21.15.79:0/3860478822 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14591 (secure 0 0 0) 0x7f0610007700 con 0x7f061c0aee30 2023-06-23T18:35:49.769 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:49.769 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:48.956106+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:35:49.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.766+0000 7f06211d6700 1 -- 172.21.15.79:0/3860478822 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f060806ace0 msgr2=0x7f060806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.766+0000 7f06211d6700 1 --2- 172.21.15.79:0/3860478822 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f060806ace0 0x7f060806d190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f060c000c00 tx=0x7f060c008030 comp rx=0 tx=0).stop 2023-06-23T18:35:49.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.766+0000 7f06211d6700 1 -- 172.21.15.79:0/3860478822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f061c0aee30 msgr2=0x7f061c0c7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.766+0000 7f06211d6700 1 --2- 172.21.15.79:0/3860478822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f061c0aee30 0x7f061c0c7b70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f06100011e0 tx=0x7f061000ca50 comp rx=0 tx=0).stop 2023-06-23T18:35:49.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.766+0000 7f06211d6700 1 -- 172.21.15.79:0/3860478822 shutdown_connections 2023-06-23T18:35:49.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.766+0000 7f06211d6700 1 --2- 172.21.15.79:0/3860478822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f061c0b1270 0x7f061c150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.766+0000 7f06211d6700 1 --2- 172.21.15.79:0/3860478822 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f060806ace0 0x7f060806d190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.766+0000 7f06211d6700 1 --2- 172.21.15.79:0/3860478822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f061c071490 0x7f061c0c4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.766+0000 7f06211d6700 1 --2- 172.21.15.79:0/3860478822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f061c0aee30 0x7f061c0c7b70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.766+0000 7f06211d6700 1 -- 172.21.15.79:0/3860478822 >> 172.21.15.79:0/3860478822 conn(0x7f061c06d660 msgr2=0x7f061c0ca920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:49.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.766+0000 7f06211d6700 1 -- 172.21.15.79:0/3860478822 shutdown_connections 2023-06-23T18:35:49.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.766+0000 7f06211d6700 1 -- 172.21.15.79:0/3860478822 wait complete. 2023-06-23T18:35:49.783 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:35:49.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:35:49.906 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.901+0000 7f9415235700 1 Processor -- start 2023-06-23T18:35:49.907 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.902+0000 7f9415235700 1 -- start start 2023-06-23T18:35:49.909 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.902+0000 7f9415235700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f94101438f0 0x7f94100b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.909 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.902+0000 7f9415235700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94100d08c0 0x7f94100d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.902+0000 7f9415235700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94100d1220 con 0x7f94100cedf0 2023-06-23T18:35:49.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.902+0000 7f9415235700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94100b0b10 con 0x7f94101438f0 2023-06-23T18:35:49.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.902+0000 7f9415235700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94100b0df0 con 0x7f94100d08c0 2023-06-23T18:35:49.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.902+0000 7f940ed9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f94100cedf0 0x7f94100b02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41982/0 (socket says 172.21.15.79:41982) 2023-06-23T18:35:49.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.902+0000 7f940ed9d700 1 -- 172.21.15.79:0/516563317 learned_addr learned my addr 172.21.15.79:0/516563317 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:49.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940e59c700 1 --2- 172.21.15.79:0/516563317 >> v2:172.21.15.155:3300/0 conn(0x7f94101438f0 0x7f94100b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940f59e700 1 --2- 172.21.15.79:0/516563317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94100d08c0 0x7f94100d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 -- 172.21.15.79:0/516563317 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035182313 0 0) 0x7f94100d1220 con 0x7f94100cedf0 2023-06-23T18:35:49.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 -- 172.21.15.79:0/516563317 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93f80035b0 con 0x7f94100cedf0 2023-06-23T18:35:49.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 -- 172.21.15.79:0/516563317 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3040453084 0 0) 0x7f93f80035b0 con 0x7f94100cedf0 2023-06-23T18:35:49.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 -- 172.21.15.79:0/516563317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94100d08c0 msgr2=0x7f94100d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:49.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 --2- 172.21.15.79:0/516563317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94100d08c0 0x7f94100d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 -- 172.21.15.79:0/516563317 >> v2:172.21.15.155:3300/0 conn(0x7f94101438f0 msgr2=0x7f94100b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:49.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 --2- 172.21.15.79:0/516563317 >> v2:172.21.15.155:3300/0 conn(0x7f94101438f0 0x7f94100b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 -- 172.21.15.79:0/516563317 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94100b10d0 con 0x7f94100cedf0 2023-06-23T18:35:49.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 -- 172.21.15.79:0/516563317 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f94040017f0 con 0x7f94100cedf0 2023-06-23T18:35:49.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 -- 172.21.15.79:0/516563317 >> v1:172.21.15.79:6789/0 conn(0x7f94100cedf0 legacy=0x7f94100b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 --2- 172.21.15.79:0/516563317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93f8003a00 0x7f93f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 --2- 172.21.15.79:0/516563317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93f8004430 0x7f93f80048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 --2- 172.21.15.79:0/516563317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94100d08c0 0x7f93f8005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 -- 172.21.15.79:0/516563317 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94100b0b10 con 0x7f93f8004430 2023-06-23T18:35:49.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 -- 172.21.15.79:0/516563317 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94100b0df0 con 0x7f93f8003a00 2023-06-23T18:35:49.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940dd9b700 1 -- 172.21.15.79:0/516563317 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f800ba10 con 0x7f94100d08c0 2023-06-23T18:35:49.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940e59c700 1 --2- 172.21.15.79:0/516563317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93f8004430 0x7f93f80048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940ed9d700 1 --2- 172.21.15.79:0/516563317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93f8003a00 0x7f93f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940f59e700 1 --2- 172.21.15.79:0/516563317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94100d08c0 0x7f93f8005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940e59c700 1 -- 172.21.15.79:0/516563317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94100d08c0 msgr2=0x7f93f8005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:49.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940e59c700 1 --2- 172.21.15.79:0/516563317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94100d08c0 0x7f93f8005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940e59c700 1 -- 172.21.15.79:0/516563317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93f8003a00 msgr2=0x7f93f8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:49.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940e59c700 1 --2- 172.21.15.79:0/516563317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93f8003a00 0x7f93f8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.903+0000 7f940e59c700 1 -- 172.21.15.79:0/516563317 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f80035b0 con 0x7f93f8004430 2023-06-23T18:35:49.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.904+0000 7f940e59c700 1 --2- 172.21.15.79:0/516563317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93f8004430 0x7f93f80048f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9400009b90 tx=0x7f93f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191f9cb770558ef7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:49.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.904+0000 7f940dd9b700 1 -- 172.21.15.79:0/516563317 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940000e6c0 con 0x7f93f8004430 2023-06-23T18:35:49.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.904+0000 7f940dd9b700 1 -- 172.21.15.79:0/516563317 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940000e820 con 0x7f93f8004430 2023-06-23T18:35:49.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.904+0000 7f940dd9b700 1 -- 172.21.15.79:0/516563317 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940000eb90 con 0x7f93f8004430 2023-06-23T18:35:49.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.904+0000 7f9415235700 1 -- 172.21.15.79:0/516563317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93f8004430 msgr2=0x7f93f80048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.904+0000 7f9415235700 1 --2- 172.21.15.79:0/516563317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93f8004430 0x7f93f80048f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9400009b90 tx=0x7f93f8005880 comp rx=0 tx=0).stop 2023-06-23T18:35:49.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.904+0000 7f940f59e700 1 -- 172.21.15.79:0/516563317 reap_dead start 2023-06-23T18:35:49.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.904+0000 7f9415235700 1 -- 172.21.15.79:0/516563317 shutdown_connections 2023-06-23T18:35:49.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.904+0000 7f9415235700 1 -- 172.21.15.79:0/516563317 >> 172.21.15.79:0/516563317 conn(0x7f941006a7b0 msgr2=0x7f9410099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:49.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.904+0000 7f9415235700 1 -- 172.21.15.79:0/516563317 shutdown_connections 2023-06-23T18:35:49.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.904+0000 7f9415235700 1 -- 172.21.15.79:0/516563317 wait complete. 2023-06-23T18:35:49.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.906+0000 7f9415235700 1 Processor -- start 2023-06-23T18:35:49.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.906+0000 7f9415235700 1 -- start start 2023-06-23T18:35:49.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.906+0000 7f9415235700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93f8004430 0x7f94101543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.906+0000 7f9415235700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9410155310 0x7f94101f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.906+0000 7f9415235700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9410156d70 0x7f9410157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.906+0000 7f9415235700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9410156880 con 0x7f9410155310 2023-06-23T18:35:49.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.906+0000 7f9415235700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94101f2f30 con 0x7f93f8004430 2023-06-23T18:35:49.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.906+0000 7f9415235700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9410154c60 con 0x7f9410156d70 2023-06-23T18:35:49.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f940e59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9410155310 0x7f94101f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f940e59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9410155310 0x7f94101f29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38954/0 (socket says 172.21.15.79:38954) 2023-06-23T18:35:49.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f940e59c700 1 -- 172.21.15.79:0/2286446122 learned_addr learned my addr 172.21.15.79:0/2286446122 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:49.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f940ed9d700 1 --2- 172.21.15.79:0/2286446122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93f8004430 0x7f94101543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f940f59e700 1 --2- 172.21.15.79:0/2286446122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9410156d70 0x7f9410157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f940e59c700 1 -- 172.21.15.79:0/2286446122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9410156d70 msgr2=0x7f9410157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f940e59c700 1 --2- 172.21.15.79:0/2286446122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9410156d70 0x7f9410157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f940e59c700 1 -- 172.21.15.79:0/2286446122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93f8004430 msgr2=0x7f94101543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:49.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f940e59c700 1 --2- 172.21.15.79:0/2286446122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93f8004430 0x7f94101543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:49.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f940e59c700 1 -- 172.21.15.79:0/2286446122 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9410155780 con 0x7f9410155310 2023-06-23T18:35:49.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f940e59c700 1 --2- 172.21.15.79:0/2286446122 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9410155310 0x7f94101f29f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f940000e4b0 tx=0x7f94000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:49.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f93f77fe700 1 -- 172.21.15.79:0/2286446122 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400021070 con 0x7f9410155310 2023-06-23T18:35:49.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f93f77fe700 1 -- 172.21.15.79:0/2286446122 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9400004540 con 0x7f9410155310 2023-06-23T18:35:49.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f9415235700 1 -- 172.21.15.79:0/2286446122 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9410157c20 con 0x7f9410155310 2023-06-23T18:35:49.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f93f77fe700 1 -- 172.21.15.79:0/2286446122 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940000e6c0 con 0x7f9410155310 2023-06-23T18:35:49.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.907+0000 7f9415235700 1 -- 172.21.15.79:0/2286446122 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94101581e0 con 0x7f9410155310 2023-06-23T18:35:49.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.908+0000 7f9415235700 1 -- 172.21.15.79:0/2286446122 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94100672c0 con 0x7f9410155310 2023-06-23T18:35:49.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.909+0000 7f93f77fe700 1 -- 172.21.15.79:0/2286446122 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9400025070 con 0x7f9410155310 2023-06-23T18:35:49.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.909+0000 7f93f77fe700 1 --2- 172.21.15.79:0/2286446122 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f93fc06ace0 0x7f93fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:49.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.909+0000 7f93f77fe700 1 -- 172.21.15.79:0/2286446122 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f940008a350 con 0x7f9410155310 2023-06-23T18:35:49.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.909+0000 7f940ed9d700 1 --2- 172.21.15.79:0/2286446122 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f93fc06ace0 0x7f93fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:49.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.909+0000 7f940ed9d700 1 --2- 172.21.15.79:0/2286446122 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f93fc06ace0 0x7f93fc06d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f94040011d0 tx=0x7f9404009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:49.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:49.913+0000 7f93f77fe700 1 -- 172.21.15.79:0/2286446122 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9400059390 con 0x7f9410155310 2023-06-23T18:35:50.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.097+0000 7f9415235700 1 -- 172.21.15.79:0/2286446122 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f94100b00a0 con 0x7f9410155310 2023-06-23T18:35:50.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.097+0000 7f93f77fe700 1 -- 172.21.15.79:0/2286446122 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9400058f20 con 0x7f9410155310 2023-06-23T18:35:50.103 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:35:50.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.099+0000 7f9415235700 1 -- 172.21.15.79:0/2286446122 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f93fc06ace0 msgr2=0x7f93fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.099+0000 7f9415235700 1 --2- 172.21.15.79:0/2286446122 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f93fc06ace0 0x7f93fc06d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f94040011d0 tx=0x7f9404009040 comp rx=0 tx=0).stop 2023-06-23T18:35:50.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.099+0000 7f9415235700 1 -- 172.21.15.79:0/2286446122 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9410155310 msgr2=0x7f94101f29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.099+0000 7f9415235700 1 --2- 172.21.15.79:0/2286446122 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9410155310 0x7f94101f29f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f940000e4b0 tx=0x7f94000047c0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.099+0000 7f9415235700 1 -- 172.21.15.79:0/2286446122 shutdown_connections 2023-06-23T18:35:50.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.099+0000 7f9415235700 1 --2- 172.21.15.79:0/2286446122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9410156d70 0x7f9410157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.099+0000 7f9415235700 1 --2- 172.21.15.79:0/2286446122 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f93fc06ace0 0x7f93fc06d190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.099+0000 7f9415235700 1 --2- 172.21.15.79:0/2286446122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93f8004430 0x7f94101543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.099+0000 7f9415235700 1 --2- 172.21.15.79:0/2286446122 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9410155310 0x7f94101f29f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.099+0000 7f9415235700 1 -- 172.21.15.79:0/2286446122 >> 172.21.15.79:0/2286446122 conn(0x7f941006a7b0 msgr2=0x7f94100b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:50.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.099+0000 7f9415235700 1 -- 172.21.15.79:0/2286446122 shutdown_connections 2023-06-23T18:35:50.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.099+0000 7f9415235700 1 -- 172.21.15.79:0/2286446122 wait complete. 2023-06-23T18:35:50.117 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:35:50.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.236+0000 7effdb328700 1 Processor -- start 2023-06-23T18:35:50.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.236+0000 7effdb328700 1 -- start start 2023-06-23T18:35:50.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.237+0000 7effdb328700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effd40cedf0 0x7effd40b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.237+0000 7effdb328700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7effd41438f0 0x7effd40b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.237+0000 7effdb328700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7effd40d08c0 0x7effd40d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.237+0000 7effdb328700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd40d1280 con 0x7effd41438f0 2023-06-23T18:35:50.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.237+0000 7effdb328700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd40b0b10 con 0x7effd40d08c0 2023-06-23T18:35:50.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.237+0000 7effdb328700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd40b0df0 con 0x7effd40cedf0 2023-06-23T18:35:50.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.237+0000 7effd88c3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7effd41438f0 0x7effd40b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.237+0000 7effd88c3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7effd41438f0 0x7effd40b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38966/0 (socket says 172.21.15.79:38966) 2023-06-23T18:35:50.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.237+0000 7effd88c3700 1 -- 172.21.15.79:0/1118268782 learned_addr learned my addr 172.21.15.79:0/1118268782 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:50.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effd98c5700 1 --2- 172.21.15.79:0/1118268782 >> v2:172.21.15.155:3300/0 conn(0x7effd40d08c0 0x7effd40d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effd90c4700 1 --2- 172.21.15.79:0/1118268782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effd40cedf0 0x7effd40b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effd88c3700 1 -- 172.21.15.79:0/1118268782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effd40cedf0 msgr2=0x7effd40b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effd88c3700 1 --2- 172.21.15.79:0/1118268782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effd40cedf0 0x7effd40b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effd88c3700 1 -- 172.21.15.79:0/1118268782 >> v2:172.21.15.155:3300/0 conn(0x7effd40d08c0 msgr2=0x7effd40d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effd88c3700 1 --2- 172.21.15.79:0/1118268782 >> v2:172.21.15.155:3300/0 conn(0x7effd40d08c0 0x7effd40d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effd88c3700 1 -- 172.21.15.79:0/1118268782 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd40b10d0 con 0x7effd41438f0 2023-06-23T18:35:50.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effd98c5700 1 --2- 172.21.15.79:0/1118268782 >> v2:172.21.15.155:3300/0 conn(0x7effd40d08c0 0x7effd40d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:35:50.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effd88c3700 1 --2- 172.21.15.79:0/1118268782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effd41438f0 0x7effd40b1bb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7effd408a5d0 tx=0x7effd0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377602ceeb8d7906 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effcbfff700 1 -- 172.21.15.79:0/1118268782 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd000b870 con 0x7effd41438f0 2023-06-23T18:35:50.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effcbfff700 1 -- 172.21.15.79:0/1118268782 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd000b9d0 con 0x7effd41438f0 2023-06-23T18:35:50.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effcbfff700 1 -- 172.21.15.79:0/1118268782 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd000bd60 con 0x7effd41438f0 2023-06-23T18:35:50.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effdb328700 1 -- 172.21.15.79:0/1118268782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effd41438f0 msgr2=0x7effd40b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effdb328700 1 --2- 172.21.15.79:0/1118268782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effd41438f0 0x7effd40b1bb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7effd408a5d0 tx=0x7effd0009bd0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effdb328700 1 -- 172.21.15.79:0/1118268782 shutdown_connections 2023-06-23T18:35:50.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effdb328700 1 --2- 172.21.15.79:0/1118268782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effd40cedf0 0x7effd40b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effdb328700 1 --2- 172.21.15.79:0/1118268782 >> v2:172.21.15.155:3300/0 conn(0x7effd40d08c0 0x7effd40d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effdb328700 1 --2- 172.21.15.79:0/1118268782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effd41438f0 0x7effd40b1bb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.238+0000 7effdb328700 1 -- 172.21.15.79:0/1118268782 >> 172.21.15.79:0/1118268782 conn(0x7effd406a7b0 msgr2=0x7effd4099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:50.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.239+0000 7effdb328700 1 -- 172.21.15.79:0/1118268782 shutdown_connections 2023-06-23T18:35:50.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.239+0000 7effdb328700 1 -- 172.21.15.79:0/1118268782 wait complete. 2023-06-23T18:35:50.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.240+0000 7effdb328700 1 Processor -- start 2023-06-23T18:35:50.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.240+0000 7effdb328700 1 -- start start 2023-06-23T18:35:50.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.240+0000 7effdb328700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effd40cedf0 0x7effd40cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.240+0000 7effdb328700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effd40d08c0 0x7effd406e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.240+0000 7effdb328700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effd41438f0 0x7effd406ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.240+0000 7effdb328700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd41f28d0 con 0x7effd40d08c0 2023-06-23T18:35:50.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.240+0000 7effdb328700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd41550c0 con 0x7effd41438f0 2023-06-23T18:35:50.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.240+0000 7effdb328700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd41f25f0 con 0x7effd40cedf0 2023-06-23T18:35:50.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effd88c3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effd40d08c0 0x7effd406e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effd88c3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effd40d08c0 0x7effd406e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38974/0 (socket says 172.21.15.79:38974) 2023-06-23T18:35:50.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effd88c3700 1 -- 172.21.15.79:0/2221615038 learned_addr learned my addr 172.21.15.79:0/2221615038 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:50.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effd98c5700 1 --2- 172.21.15.79:0/2221615038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effd41438f0 0x7effd406ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effd90c4700 1 --2- 172.21.15.79:0/2221615038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effd40cedf0 0x7effd40cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effd88c3700 1 -- 172.21.15.79:0/2221615038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effd40cedf0 msgr2=0x7effd40cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effd88c3700 1 --2- 172.21.15.79:0/2221615038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effd40cedf0 0x7effd40cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effd88c3700 1 -- 172.21.15.79:0/2221615038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effd41438f0 msgr2=0x7effd406ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effd88c3700 1 --2- 172.21.15.79:0/2221615038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effd41438f0 0x7effd406ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effd88c3700 1 -- 172.21.15.79:0/2221615038 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd0009550 con 0x7effd40d08c0 2023-06-23T18:35:50.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effd88c3700 1 --2- 172.21.15.79:0/2221615038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effd40d08c0 0x7effd406e680 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7effd0000c00 tx=0x7effd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effc9ffb700 1 -- 172.21.15.79:0/2221615038 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd00036a0 con 0x7effd40d08c0 2023-06-23T18:35:50.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effc9ffb700 1 -- 172.21.15.79:0/2221615038 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd0003800 con 0x7effd40d08c0 2023-06-23T18:35:50.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effc9ffb700 1 -- 172.21.15.79:0/2221615038 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0003b90 con 0x7effd40d08c0 2023-06-23T18:35:50.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effdb328700 1 -- 172.21.15.79:0/2221615038 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd41f5000 con 0x7effd40d08c0 2023-06-23T18:35:50.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.241+0000 7effdb328700 1 -- 172.21.15.79:0/2221615038 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd41f55c0 con 0x7effd40d08c0 2023-06-23T18:35:50.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.242+0000 7effdb328700 1 -- 172.21.15.79:0/2221615038 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd40672c0 con 0x7effd40d08c0 2023-06-23T18:35:50.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.243+0000 7effc9ffb700 1 -- 172.21.15.79:0/2221615038 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7effd0003cf0 con 0x7effd40d08c0 2023-06-23T18:35:50.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.243+0000 7effc9ffb700 1 --2- 172.21.15.79:0/2221615038 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7effb806ace0 0x7effb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.243+0000 7effc9ffb700 1 -- 172.21.15.79:0/2221615038 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7effd008a7d0 con 0x7effd40d08c0 2023-06-23T18:35:50.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.243+0000 7effd90c4700 1 --2- 172.21.15.79:0/2221615038 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7effb806ace0 0x7effb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.243+0000 7effd90c4700 1 --2- 172.21.15.79:0/2221615038 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7effb806ace0 0x7effb806d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7effd40b47b0 tx=0x7effc4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.247+0000 7effc9ffb700 1 -- 172.21.15.79:0/2221615038 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7effd0055c60 con 0x7effd40d08c0 2023-06-23T18:35:50.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.432+0000 7effdb328700 1 -- 172.21.15.79:0/2221615038 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7effd40c41f0 con 0x7effd40d08c0 2023-06-23T18:35:50.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.433+0000 7effc9ffb700 1 -- 172.21.15.79:0/2221615038 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7effd0059280 con 0x7effd40d08c0 2023-06-23T18:35:50.438 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:35:50.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.435+0000 7effdb328700 1 -- 172.21.15.79:0/2221615038 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7effb806ace0 msgr2=0x7effb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.435+0000 7effdb328700 1 --2- 172.21.15.79:0/2221615038 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7effb806ace0 0x7effb806d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7effd40b47b0 tx=0x7effc4008040 comp rx=0 tx=0).stop 2023-06-23T18:35:50.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.435+0000 7effdb328700 1 -- 172.21.15.79:0/2221615038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effd40d08c0 msgr2=0x7effd406e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.435+0000 7effdb328700 1 --2- 172.21.15.79:0/2221615038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effd40d08c0 0x7effd406e680 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7effd0000c00 tx=0x7effd0004750 comp rx=0 tx=0).stop 2023-06-23T18:35:50.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.435+0000 7effdb328700 1 -- 172.21.15.79:0/2221615038 shutdown_connections 2023-06-23T18:35:50.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.435+0000 7effdb328700 1 --2- 172.21.15.79:0/2221615038 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7effb806ace0 0x7effb806d190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.435+0000 7effdb328700 1 --2- 172.21.15.79:0/2221615038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effd41438f0 0x7effd406ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.435+0000 7effdb328700 1 --2- 172.21.15.79:0/2221615038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effd40cedf0 0x7effd40cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.435+0000 7effdb328700 1 --2- 172.21.15.79:0/2221615038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effd40d08c0 0x7effd406e680 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.435+0000 7effdb328700 1 -- 172.21.15.79:0/2221615038 >> 172.21.15.79:0/2221615038 conn(0x7effd406a7b0 msgr2=0x7effd40ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:50.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.435+0000 7effdb328700 1 -- 172.21.15.79:0/2221615038 shutdown_connections 2023-06-23T18:35:50.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.435+0000 7effdb328700 1 -- 172.21.15.79:0/2221615038 wait complete. 2023-06-23T18:35:50.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-23T18:35:50.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.566+0000 7f5d79cdf700 1 Processor -- start 2023-06-23T18:35:50.572 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.566+0000 7f5d79cdf700 1 -- start start 2023-06-23T18:35:50.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.567+0000 7f5d79cdf700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5d74071490 0x7f5d74071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.567+0000 7f5d79cdf700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d740b1270 0x7f5d74071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.567+0000 7f5d79cdf700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d741608f0 con 0x7f5d740aee30 2023-06-23T18:35:50.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.567+0000 7f5d79cdf700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d740722e0 con 0x7f5d74071490 2023-06-23T18:35:50.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.567+0000 7f5d79cdf700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d74160e20 con 0x7f5d740b1270 2023-06-23T18:35:50.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.567+0000 7f5d73fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5d740aee30 0x7f5d74069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42026/0 (socket says 172.21.15.79:42026) 2023-06-23T18:35:50.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.567+0000 7f5d73fff700 1 -- 172.21.15.79:0/772324621 learned_addr learned my addr 172.21.15.79:0/772324621 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:50.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.567+0000 7f5d737fe700 1 --2- 172.21.15.79:0/772324621 >> v2:172.21.15.155:3300/0 conn(0x7f5d74071490 0x7f5d74071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.567+0000 7f5d72ffd700 1 --2- 172.21.15.79:0/772324621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d740b1270 0x7f5d74071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.567+0000 7f5d727fc700 1 -- 172.21.15.79:0/772324621 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881194989 0 0) 0x7f5d741608f0 con 0x7f5d740aee30 2023-06-23T18:35:50.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.567+0000 7f5d727fc700 1 -- 172.21.15.79:0/772324621 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d5c0035b0 con 0x7f5d740aee30 2023-06-23T18:35:50.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 -- 172.21.15.79:0/772324621 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 166918611 0 0) 0x7f5d5c0035b0 con 0x7f5d740aee30 2023-06-23T18:35:50.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 -- 172.21.15.79:0/772324621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d740b1270 msgr2=0x7f5d74071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 --2- 172.21.15.79:0/772324621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d740b1270 0x7f5d74071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 -- 172.21.15.79:0/772324621 >> v2:172.21.15.155:3300/0 conn(0x7f5d74071490 msgr2=0x7f5d74071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 --2- 172.21.15.79:0/772324621 >> v2:172.21.15.155:3300/0 conn(0x7f5d74071490 0x7f5d74071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 -- 172.21.15.79:0/772324621 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d740cd4f0 con 0x7f5d740aee30 2023-06-23T18:35:50.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 -- 172.21.15.79:0/772324621 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f5d6c0019f0 con 0x7f5d740aee30 2023-06-23T18:35:50.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 -- 172.21.15.79:0/772324621 >> v1:172.21.15.79:6789/0 conn(0x7f5d740aee30 legacy=0x7f5d74069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 --2- 172.21.15.79:0/772324621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c003a00 0x7f5d5c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 --2- 172.21.15.79:0/772324621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d5c004430 0x7f5d5c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 --2- 172.21.15.79:0/772324621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d740b1270 0x7f5d5c005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 -- 172.21.15.79:0/772324621 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d740722e0 con 0x7f5d5c003a00 2023-06-23T18:35:50.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 -- 172.21.15.79:0/772324621 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d74160e20 con 0x7f5d5c004430 2023-06-23T18:35:50.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 -- 172.21.15.79:0/772324621 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c00ba10 con 0x7f5d740b1270 2023-06-23T18:35:50.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d737fe700 1 --2- 172.21.15.79:0/772324621 >> v2:172.21.15.155:3300/0 conn(0x7f5d74071490 0x7f5d74071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:50.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d72ffd700 1 --2- 172.21.15.79:0/772324621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d5c004430 0x7f5d5c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d737fe700 1 --2- 172.21.15.79:0/772324621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c003a00 0x7f5d5c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d73fff700 1 --2- 172.21.15.79:0/772324621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d740b1270 0x7f5d5c005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d737fe700 1 -- 172.21.15.79:0/772324621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d740b1270 msgr2=0x7f5d5c005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d737fe700 1 --2- 172.21.15.79:0/772324621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d740b1270 0x7f5d5c005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d737fe700 1 -- 172.21.15.79:0/772324621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d5c004430 msgr2=0x7f5d5c0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d737fe700 1 --2- 172.21.15.79:0/772324621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d5c004430 0x7f5d5c0048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d737fe700 1 -- 172.21.15.79:0/772324621 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c0035b0 con 0x7f5d5c003a00 2023-06-23T18:35:50.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d737fe700 1 --2- 172.21.15.79:0/772324621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c003a00 0x7f5d5c003e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009d40 tx=0x7f5d5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78286d9d6bf5271 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.568+0000 7f5d727fc700 1 -- 172.21.15.79:0/772324621 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68009e90 con 0x7f5d5c003a00 2023-06-23T18:35:50.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.569+0000 7f5d727fc700 1 -- 172.21.15.79:0/772324621 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6800e9a0 con 0x7f5d5c003a00 2023-06-23T18:35:50.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.569+0000 7f5d727fc700 1 -- 172.21.15.79:0/772324621 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6800ed10 con 0x7f5d5c003a00 2023-06-23T18:35:50.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.569+0000 7f5d79cdf700 1 -- 172.21.15.79:0/772324621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c003a00 msgr2=0x7f5d5c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.569+0000 7f5d79cdf700 1 --2- 172.21.15.79:0/772324621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c003a00 0x7f5d5c003e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009d40 tx=0x7f5d5c005880 comp rx=0 tx=0).stop 2023-06-23T18:35:50.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.569+0000 7f5d73fff700 1 -- 172.21.15.79:0/772324621 reap_dead start 2023-06-23T18:35:50.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.569+0000 7f5d79cdf700 1 -- 172.21.15.79:0/772324621 shutdown_connections 2023-06-23T18:35:50.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.569+0000 7f5d79cdf700 1 -- 172.21.15.79:0/772324621 >> 172.21.15.79:0/772324621 conn(0x7f5d7406d660 msgr2=0x7f5d7406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:50.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.569+0000 7f5d79cdf700 1 -- 172.21.15.79:0/772324621 shutdown_connections 2023-06-23T18:35:50.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.569+0000 7f5d79cdf700 1 -- 172.21.15.79:0/772324621 wait complete. 2023-06-23T18:35:50.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.570+0000 7f5d79cdf700 1 Processor -- start 2023-06-23T18:35:50.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.571+0000 7f5d79cdf700 1 -- start start 2023-06-23T18:35:50.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.571+0000 7f5d79cdf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c003a00 0x7f5d74153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.571+0000 7f5d79cdf700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d740c5230 0x7f5d740b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.571+0000 7f5d79cdf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d740c56a0 0x7f5d740abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.571+0000 7f5d79cdf700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d741543a0 con 0x7f5d5c003a00 2023-06-23T18:35:50.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.571+0000 7f5d79cdf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d740c4190 con 0x7f5d740c56a0 2023-06-23T18:35:50.591 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.571+0000 7f5d79cdf700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d740c4470 con 0x7f5d740c5230 2023-06-23T18:35:50.591 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.571+0000 7f5d737fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c003a00 0x7f5d74153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.591 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.571+0000 7f5d737fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c003a00 0x7f5d74153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38992/0 (socket says 172.21.15.79:38992) 2023-06-23T18:35:50.591 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.571+0000 7f5d737fe700 1 -- 172.21.15.79:0/717810591 learned_addr learned my addr 172.21.15.79:0/717810591 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:50.591 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.571+0000 7f5d72ffd700 1 --2- 172.21.15.79:0/717810591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d740c5230 0x7f5d740b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.571+0000 7f5d73fff700 1 --2- 172.21.15.79:0/717810591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d740c56a0 0x7f5d740abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.572+0000 7f5d737fe700 1 -- 172.21.15.79:0/717810591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d740c5230 msgr2=0x7f5d740b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.572+0000 7f5d737fe700 1 --2- 172.21.15.79:0/717810591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d740c5230 0x7f5d740b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.572+0000 7f5d737fe700 1 -- 172.21.15.79:0/717810591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d740c56a0 msgr2=0x7f5d740abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.572+0000 7f5d737fe700 1 --2- 172.21.15.79:0/717810591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d740c56a0 0x7f5d740abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.572+0000 7f5d737fe700 1 -- 172.21.15.79:0/717810591 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d740c4750 con 0x7f5d5c003a00 2023-06-23T18:35:50.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.572+0000 7f5d737fe700 1 --2- 172.21.15.79:0/717810591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c003a00 0x7f5d74153e60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c005770 tx=0x7f5d680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.572+0000 7f5d5bfff700 1 -- 172.21.15.79:0/717810591 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68021070 con 0x7f5d5c003a00 2023-06-23T18:35:50.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.572+0000 7f5d5bfff700 1 -- 172.21.15.79:0/717810591 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d68009e90 con 0x7f5d5c003a00 2023-06-23T18:35:50.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.572+0000 7f5d79cdf700 1 -- 172.21.15.79:0/717810591 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d740c4a30 con 0x7f5d5c003a00 2023-06-23T18:35:50.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.572+0000 7f5d5bfff700 1 -- 172.21.15.79:0/717810591 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68003af0 con 0x7f5d5c003a00 2023-06-23T18:35:50.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.572+0000 7f5d79cdf700 1 -- 172.21.15.79:0/717810591 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d740c3e40 con 0x7f5d5c003a00 2023-06-23T18:35:50.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.573+0000 7f5d79cdf700 1 -- 172.21.15.79:0/717810591 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d740724a0 con 0x7f5d5c003a00 2023-06-23T18:35:50.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.573+0000 7f5d5bfff700 1 -- 172.21.15.79:0/717810591 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d68003c50 con 0x7f5d5c003a00 2023-06-23T18:35:50.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.574+0000 7f5d5bfff700 1 --2- 172.21.15.79:0/717810591 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5d6006ace0 0x7f5d6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.574+0000 7f5d5bfff700 1 -- 172.21.15.79:0/717810591 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5d6808a1f0 con 0x7f5d5c003a00 2023-06-23T18:35:50.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.574+0000 7f5d72ffd700 1 --2- 172.21.15.79:0/717810591 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5d6006ace0 0x7f5d6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.574+0000 7f5d72ffd700 1 --2- 172.21.15.79:0/717810591 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5d6006ace0 0x7f5d6006d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5d740c3de0 tx=0x7f5d64009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.578+0000 7f5d5bfff700 1 -- 172.21.15.79:0/717810591 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5d680557a0 con 0x7f5d5c003a00 2023-06-23T18:35:50.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.768+0000 7f5d79cdf700 1 -- 172.21.15.79:0/717810591 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5d740b5770 con 0x7f5d5c003a00 2023-06-23T18:35:50.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.769+0000 7f5d5bfff700 1 -- 172.21.15.79:0/717810591 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+126 (secure 0 0 0) 0x7f5d68058dc0 con 0x7f5d5c003a00 2023-06-23T18:35:50.774 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:50.775 INFO:teuthology.orchestra.run.smithi079.stdout:[ 2023-06-23T18:35:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:35:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: "poolnum": 1, 2023-06-23T18:35:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: "poolname": "rbd" 2023-06-23T18:35:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:35:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:35:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: "poolnum": 2, 2023-06-23T18:35:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: "poolname": ".mgr" 2023-06-23T18:35:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:35:50.776 INFO:teuthology.orchestra.run.smithi079.stdout:] 2023-06-23T18:35:50.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.771+0000 7f5d79cdf700 1 -- 172.21.15.79:0/717810591 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5d6006ace0 msgr2=0x7f5d6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.771+0000 7f5d79cdf700 1 --2- 172.21.15.79:0/717810591 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5d6006ace0 0x7f5d6006d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5d740c3de0 tx=0x7f5d64009210 comp rx=0 tx=0).stop 2023-06-23T18:35:50.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.771+0000 7f5d79cdf700 1 -- 172.21.15.79:0/717810591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c003a00 msgr2=0x7f5d74153e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.771+0000 7f5d79cdf700 1 --2- 172.21.15.79:0/717810591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c003a00 0x7f5d74153e60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c005770 tx=0x7f5d680047c0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.771+0000 7f5d79cdf700 1 -- 172.21.15.79:0/717810591 shutdown_connections 2023-06-23T18:35:50.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.771+0000 7f5d79cdf700 1 --2- 172.21.15.79:0/717810591 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5d6006ace0 0x7f5d6006d190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.771+0000 7f5d79cdf700 1 --2- 172.21.15.79:0/717810591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d740c56a0 0x7f5d740abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.771+0000 7f5d79cdf700 1 --2- 172.21.15.79:0/717810591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c003a00 0x7f5d74153e60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.771+0000 7f5d79cdf700 1 --2- 172.21.15.79:0/717810591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d740c5230 0x7f5d740b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.771+0000 7f5d79cdf700 1 -- 172.21.15.79:0/717810591 >> 172.21.15.79:0/717810591 conn(0x7f5d7406d660 msgr2=0x7f5d740c9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:50.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.771+0000 7f5d79cdf700 1 -- 172.21.15.79:0/717810591 shutdown_connections 2023-06-23T18:35:50.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.771+0000 7f5d79cdf700 1 -- 172.21.15.79:0/717810591 wait complete. 2023-06-23T18:35:50.788 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-23T18:35:50.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-23T18:35:50.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.908+0000 7eff8af29700 1 Processor -- start 2023-06-23T18:35:50.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.908+0000 7eff8af29700 1 -- start start 2023-06-23T18:35:50.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.908+0000 7eff8af29700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff840cb870 0x7eff84160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.908+0000 7eff8af29700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7eff840cf1f0 0x7eff8415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.908+0000 7eff8af29700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7eff84069e90 0x7eff8406a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.908+0000 7eff8af29700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8415cac0 con 0x7eff84069e90 2023-06-23T18:35:50.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.908+0000 7eff8af29700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8406a850 con 0x7eff840cf1f0 2023-06-23T18:35:50.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.908+0000 7eff8af29700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff84160f50 con 0x7eff840cb870 2023-06-23T18:35:50.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff894c6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7eff84069e90 0x7eff8406a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff894c6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7eff84069e90 0x7eff8406a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39006/0 (socket says 172.21.15.79:39006) 2023-06-23T18:35:50.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff894c6700 1 -- 172.21.15.79:0/2566774105 learned_addr learned my addr 172.21.15.79:0/2566774105 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:50.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff88cc5700 1 --2- 172.21.15.79:0/2566774105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff840cb870 0x7eff84160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff83fff700 1 --2- 172.21.15.79:0/2566774105 >> v2:172.21.15.155:3300/0 conn(0x7eff840cf1f0 0x7eff8415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff894c6700 1 -- 172.21.15.79:0/2566774105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff840cb870 msgr2=0x7eff84160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff894c6700 1 --2- 172.21.15.79:0/2566774105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff840cb870 0x7eff84160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff894c6700 1 -- 172.21.15.79:0/2566774105 >> v2:172.21.15.155:3300/0 conn(0x7eff840cf1f0 msgr2=0x7eff8415d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff894c6700 1 --2- 172.21.15.79:0/2566774105 >> v2:172.21.15.155:3300/0 conn(0x7eff840cf1f0 0x7eff8415d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff894c6700 1 -- 172.21.15.79:0/2566774105 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff840cd410 con 0x7eff84069e90 2023-06-23T18:35:50.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff83fff700 1 --2- 172.21.15.79:0/2566774105 >> v2:172.21.15.155:3300/0 conn(0x7eff840cf1f0 0x7eff8415d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:35:50.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff894c6700 1 --2- 172.21.15.79:0/2566774105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff84069e90 0x7eff8406a310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7eff8406e9c0 tx=0x7eff7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3871b16afea7c720 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff837fe700 1 -- 172.21.15.79:0/2566774105 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c00e070 con 0x7eff84069e90 2023-06-23T18:35:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff837fe700 1 -- 172.21.15.79:0/2566774105 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c00b9c0 con 0x7eff84069e90 2023-06-23T18:35:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.909+0000 7eff837fe700 1 -- 172.21.15.79:0/2566774105 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c00bd70 con 0x7eff84069e90 2023-06-23T18:35:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.910+0000 7eff8af29700 1 -- 172.21.15.79:0/2566774105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff84069e90 msgr2=0x7eff8406a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.910+0000 7eff8af29700 1 --2- 172.21.15.79:0/2566774105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff84069e90 0x7eff8406a310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7eff8406e9c0 tx=0x7eff7c00b330 comp rx=0 tx=0).stop 2023-06-23T18:35:50.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.910+0000 7eff8af29700 1 -- 172.21.15.79:0/2566774105 shutdown_connections 2023-06-23T18:35:50.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.910+0000 7eff8af29700 1 --2- 172.21.15.79:0/2566774105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff840cb870 0x7eff84160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.910+0000 7eff8af29700 1 --2- 172.21.15.79:0/2566774105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff84069e90 0x7eff8406a310 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.910+0000 7eff8af29700 1 --2- 172.21.15.79:0/2566774105 >> v2:172.21.15.155:3300/0 conn(0x7eff840cf1f0 0x7eff8415d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.910+0000 7eff8af29700 1 -- 172.21.15.79:0/2566774105 >> 172.21.15.79:0/2566774105 conn(0x7eff8406e200 msgr2=0x7eff840cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:50.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.910+0000 7eff8af29700 1 -- 172.21.15.79:0/2566774105 shutdown_connections 2023-06-23T18:35:50.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.910+0000 7eff8af29700 1 -- 172.21.15.79:0/2566774105 wait complete. 2023-06-23T18:35:50.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.911+0000 7eff8af29700 1 Processor -- start 2023-06-23T18:35:50.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.911+0000 7eff8af29700 1 -- start start 2023-06-23T18:35:50.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff8af29700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff84069e90 0x7eff8415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff8af29700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff840cb870 0x7eff8415be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff8af29700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff840cf1f0 0x7eff840b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff8af29700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8415eb60 con 0x7eff84069e90 2023-06-23T18:35:50.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff8af29700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8415c370 con 0x7eff840cf1f0 2023-06-23T18:35:50.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff8af29700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff840ca760 con 0x7eff840cb870 2023-06-23T18:35:50.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff88cc5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff84069e90 0x7eff8415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff88cc5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff84069e90 0x7eff8415dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39008/0 (socket says 172.21.15.79:39008) 2023-06-23T18:35:50.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff88cc5700 1 -- 172.21.15.79:0/593378111 learned_addr learned my addr 172.21.15.79:0/593378111 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:50.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff894c6700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff840cf1f0 0x7eff840b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff83fff700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff840cb870 0x7eff8415be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff88cc5700 1 -- 172.21.15.79:0/593378111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff840cb870 msgr2=0x7eff8415be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff88cc5700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff840cb870 0x7eff8415be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff88cc5700 1 -- 172.21.15.79:0/593378111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff840cf1f0 msgr2=0x7eff840b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff88cc5700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff840cf1f0 0x7eff840b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff88cc5700 1 -- 172.21.15.79:0/593378111 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7c009b30 con 0x7eff84069e90 2023-06-23T18:35:50.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff83fff700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff840cb870 0x7eff8415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:35:50.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff88cc5700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff84069e90 0x7eff8415dbc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7eff7800e810 tx=0x7eff7800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff817fa700 1 -- 172.21.15.79:0/593378111 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff78013610 con 0x7eff84069e90 2023-06-23T18:35:50.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff817fa700 1 -- 172.21.15.79:0/593378111 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff78013770 con 0x7eff84069e90 2023-06-23T18:35:50.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff8af29700 1 -- 172.21.15.79:0/593378111 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8415e2c0 con 0x7eff84069e90 2023-06-23T18:35:50.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.912+0000 7eff817fa700 1 -- 172.21.15.79:0/593378111 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff78013b00 con 0x7eff84069e90 2023-06-23T18:35:50.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.913+0000 7eff8af29700 1 -- 172.21.15.79:0/593378111 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8415e880 con 0x7eff84069e90 2023-06-23T18:35:50.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.913+0000 7eff8af29700 1 -- 172.21.15.79:0/593378111 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff840672c0 con 0x7eff84069e90 2023-06-23T18:35:50.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.914+0000 7eff817fa700 1 -- 172.21.15.79:0/593378111 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7eff78013c60 con 0x7eff84069e90 2023-06-23T18:35:50.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.914+0000 7eff817fa700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7eff6806ace0 0x7eff6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.914+0000 7eff817fa700 1 -- 172.21.15.79:0/593378111 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7eff78088cf0 con 0x7eff84069e90 2023-06-23T18:35:50.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.914+0000 7eff83fff700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7eff6806ace0 0x7eff6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.915+0000 7eff83fff700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7eff6806ace0 0x7eff6806d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7eff840b4ba0 tx=0x7eff70008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:50.919+0000 7eff817fa700 1 -- 172.21.15.79:0/593378111 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7eff7808d070 con 0x7eff84069e90 2023-06-23T18:35:51.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:51.106+0000 7eff8af29700 1 -- 172.21.15.79:0/593378111 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7eff840ccc70 con 0x7eff84069e90 2023-06-23T18:35:52.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.110+0000 7eff817fa700 1 -- 172.21.15.79:0/593378111 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7eff780210b0 con 0x7eff84069e90 2023-06-23T18:35:52.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.115+0000 7eff8af29700 1 -- 172.21.15.79:0/593378111 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7eff6806ace0 msgr2=0x7eff6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:52.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.115+0000 7eff8af29700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7eff6806ace0 0x7eff6806d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7eff840b4ba0 tx=0x7eff70008040 comp rx=0 tx=0).stop 2023-06-23T18:35:52.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.115+0000 7eff8af29700 1 -- 172.21.15.79:0/593378111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff84069e90 msgr2=0x7eff8415dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:52.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.115+0000 7eff8af29700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff84069e90 0x7eff8415dbc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7eff7800e810 tx=0x7eff7800ebd0 comp rx=0 tx=0).stop 2023-06-23T18:35:52.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.116+0000 7eff8af29700 1 -- 172.21.15.79:0/593378111 shutdown_connections 2023-06-23T18:35:52.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.116+0000 7eff8af29700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7eff6806ace0 0x7eff6806d190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:52.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.116+0000 7eff8af29700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff840cf1f0 0x7eff840b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:52.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.116+0000 7eff8af29700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff84069e90 0x7eff8415dbc0 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7eff7800e810 tx=0x7eff7800ebd0 comp rx=0 tx=0).stop 2023-06-23T18:35:52.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.116+0000 7eff8af29700 1 --2- 172.21.15.79:0/593378111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff840cb870 0x7eff8415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:52.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.116+0000 7eff8af29700 1 -- 172.21.15.79:0/593378111 >> 172.21.15.79:0/593378111 conn(0x7eff8406e200 msgr2=0x7eff8406eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:52.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.117+0000 7eff8af29700 1 -- 172.21.15.79:0/593378111 shutdown_connections 2023-06-23T18:35:52.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.117+0000 7eff8af29700 1 -- 172.21.15.79:0/593378111 wait complete. 2023-06-23T18:35:52.123 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_metadata' created 2023-06-23T18:35:52.135 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-23T18:35:52.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.261+0000 7f836b500700 1 Processor -- start 2023-06-23T18:35:52.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.261+0000 7f836b500700 1 -- start start 2023-06-23T18:35:52.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.261+0000 7f836b500700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f83640cec30 0x7f836415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:52.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.261+0000 7f836b500700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f836415e2c0 0x7f83640cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:52.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.261+0000 7f836b500700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f836415eab0 con 0x7f836415a0a0 2023-06-23T18:35:52.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f836b500700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836415ed90 con 0x7f83640cec30 2023-06-23T18:35:52.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f836b500700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83640cb140 con 0x7f836415e2c0 2023-06-23T18:35:52.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f8368a9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f83640cec30 0x7f836415dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:52.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f8368a9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f83640cec30 0x7f836415dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39018/0 (socket says 172.21.15.79:39018) 2023-06-23T18:35:52.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f836929c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f836415a0a0 0x7f83640ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42058/0 (socket says 172.21.15.79:42058) 2023-06-23T18:35:52.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f8368a9b700 1 -- 172.21.15.79:0/3081787781 learned_addr learned my addr 172.21.15.79:0/3081787781 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:52.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f8369a9d700 1 --2- 172.21.15.79:0/3081787781 >> v2:172.21.15.155:3300/0 conn(0x7f836415e2c0 0x7f83640cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:52.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f8368a9b700 1 -- 172.21.15.79:0/3081787781 >> v2:172.21.15.155:3300/0 conn(0x7f836415e2c0 msgr2=0x7f83640cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:52.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f8368a9b700 1 --2- 172.21.15.79:0/3081787781 >> v2:172.21.15.155:3300/0 conn(0x7f836415e2c0 0x7f83640cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:52.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f8368a9b700 1 -- 172.21.15.79:0/3081787781 >> v1:172.21.15.79:6789/0 conn(0x7f836415a0a0 legacy=0x7f83640ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:52.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f8368a9b700 1 -- 172.21.15.79:0/3081787781 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83640cb420 con 0x7f83640cec30 2023-06-23T18:35:52.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f835bfff700 1 -- 172.21.15.79:0/3081787781 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453772447 0 0) 0x7f836415eab0 con 0x7f836415a0a0 2023-06-23T18:35:52.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f8368a9b700 1 --2- 172.21.15.79:0/3081787781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83640cec30 0x7f836415dd80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f83640723f0 tx=0x7f8360009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c017d1c3b9c923 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:52.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f835bfff700 1 -- 172.21.15.79:0/3081787781 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836000b870 con 0x7f83640cec30 2023-06-23T18:35:52.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f835bfff700 1 -- 172.21.15.79:0/3081787781 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836000b9d0 con 0x7f83640cec30 2023-06-23T18:35:52.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.262+0000 7f835bfff700 1 -- 172.21.15.79:0/3081787781 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836000bd60 con 0x7f83640cec30 2023-06-23T18:35:52.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.263+0000 7f836b500700 1 -- 172.21.15.79:0/3081787781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83640cec30 msgr2=0x7f836415dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:52.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.263+0000 7f836b500700 1 --2- 172.21.15.79:0/3081787781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83640cec30 0x7f836415dd80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f83640723f0 tx=0x7f8360009bd0 comp rx=0 tx=0).stop 2023-06-23T18:35:52.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.263+0000 7f836b500700 1 -- 172.21.15.79:0/3081787781 shutdown_connections 2023-06-23T18:35:52.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.263+0000 7f836b500700 1 --2- 172.21.15.79:0/3081787781 >> v2:172.21.15.155:3300/0 conn(0x7f836415e2c0 0x7f83640cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:52.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.263+0000 7f836b500700 1 --2- 172.21.15.79:0/3081787781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83640cec30 0x7f836415dd80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:52.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.263+0000 7f836b500700 1 -- 172.21.15.79:0/3081787781 >> 172.21.15.79:0/3081787781 conn(0x7f8364099cc0 msgr2=0x7f8364089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:52.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.263+0000 7f836b500700 1 -- 172.21.15.79:0/3081787781 shutdown_connections 2023-06-23T18:35:52.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.263+0000 7f836b500700 1 -- 172.21.15.79:0/3081787781 wait complete. 2023-06-23T18:35:52.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.264+0000 7f836b500700 1 Processor -- start 2023-06-23T18:35:52.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.264+0000 7f836b500700 1 -- start start 2023-06-23T18:35:52.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f836b500700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83640cec30 0x7f83640c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:52.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f836b500700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f836415a0a0 0x7f83641e5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:52.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f836b500700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f836415e2c0 0x7f83640c4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:52.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f836b500700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83640b3780 con 0x7f836415a0a0 2023-06-23T18:35:52.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f836b500700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83641e5ba0 con 0x7f836415e2c0 2023-06-23T18:35:52.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f836b500700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83641e5e80 con 0x7f83640cec30 2023-06-23T18:35:52.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f8368a9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f836415a0a0 0x7f83641e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:52.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f8368a9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f836415a0a0 0x7f83641e5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39024/0 (socket says 172.21.15.79:39024) 2023-06-23T18:35:52.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f8368a9b700 1 -- 172.21.15.79:0/314819541 learned_addr learned my addr 172.21.15.79:0/314819541 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:52.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f8368a9b700 1 -- 172.21.15.79:0/314819541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83640cec30 msgr2=0x7f83640c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:52.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f8368a9b700 1 --2- 172.21.15.79:0/314819541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83640cec30 0x7f83640c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:52.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f8368a9b700 1 -- 172.21.15.79:0/314819541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f836415e2c0 msgr2=0x7f83640c4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:52.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f8369a9d700 1 --2- 172.21.15.79:0/314819541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f836415e2c0 0x7f83640c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:52.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f8368a9b700 1 --2- 172.21.15.79:0/314819541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f836415e2c0 0x7f83640c4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:52.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f8368a9b700 1 -- 172.21.15.79:0/314819541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8360009550 con 0x7f836415a0a0 2023-06-23T18:35:52.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f8368a9b700 1 --2- 172.21.15.79:0/314819541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f836415a0a0 0x7f83641e5090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8360009c10 tx=0x7f8360004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:52.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f8359ffb700 1 -- 172.21.15.79:0/314819541 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360004250 con 0x7f836415a0a0 2023-06-23T18:35:52.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f8359ffb700 1 -- 172.21.15.79:0/314819541 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836001f070 con 0x7f836415a0a0 2023-06-23T18:35:52.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f836b500700 1 -- 172.21.15.79:0/314819541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83641e6160 con 0x7f836415a0a0 2023-06-23T18:35:52.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.265+0000 7f8359ffb700 1 -- 172.21.15.79:0/314819541 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83600045d0 con 0x7f836415a0a0 2023-06-23T18:35:52.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.266+0000 7f836b500700 1 -- 172.21.15.79:0/314819541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83640c60c0 con 0x7f836415a0a0 2023-06-23T18:35:52.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.267+0000 7f836b500700 1 -- 172.21.15.79:0/314819541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83641e6350 con 0x7f836415a0a0 2023-06-23T18:35:52.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.267+0000 7f8359ffb700 1 -- 172.21.15.79:0/314819541 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8360021030 con 0x7f836415a0a0 2023-06-23T18:35:52.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.267+0000 7f8359ffb700 1 --2- 172.21.15.79:0/314819541 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f834806ace0 0x7f834806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:52.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.268+0000 7f8359ffb700 1 -- 172.21.15.79:0/314819541 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f836008a600 con 0x7f836415a0a0 2023-06-23T18:35:52.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.268+0000 7f836929c700 1 --2- 172.21.15.79:0/314819541 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f834806ace0 0x7f834806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:52.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.268+0000 7f836929c700 1 --2- 172.21.15.79:0/314819541 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f834806ace0 0x7f834806d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8354001020 tx=0x7f8354007910 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:52.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.272+0000 7f8359ffb700 1 -- 172.21.15.79:0/314819541 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8360055920 con 0x7f836415a0a0 2023-06-23T18:35:52.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:52.460+0000 7f836b500700 1 -- 172.21.15.79:0/314819541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f83640672c0 con 0x7f836415a0a0 2023-06-23T18:35:53.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.112+0000 7f8359ffb700 1 -- 172.21.15.79:0/314819541 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8360058f40 con 0x7f836415a0a0 2023-06-23T18:35:53.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.117+0000 7f836b500700 1 -- 172.21.15.79:0/314819541 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f834806ace0 msgr2=0x7f834806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.117+0000 7f836b500700 1 --2- 172.21.15.79:0/314819541 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f834806ace0 0x7f834806d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8354001020 tx=0x7f8354007910 comp rx=0 tx=0).stop 2023-06-23T18:35:53.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.117+0000 7f836b500700 1 -- 172.21.15.79:0/314819541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f836415a0a0 msgr2=0x7f83641e5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.117+0000 7f836b500700 1 --2- 172.21.15.79:0/314819541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f836415a0a0 0x7f83641e5090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8360009c10 tx=0x7f8360004750 comp rx=0 tx=0).stop 2023-06-23T18:35:53.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.119+0000 7f836b500700 1 -- 172.21.15.79:0/314819541 shutdown_connections 2023-06-23T18:35:53.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.119+0000 7f836b500700 1 --2- 172.21.15.79:0/314819541 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f834806ace0 0x7f834806d190 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8354001020 tx=0x7f8354007910 comp rx=0 tx=0).stop 2023-06-23T18:35:53.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.119+0000 7f836b500700 1 --2- 172.21.15.79:0/314819541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f836415e2c0 0x7f83640c4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.119+0000 7f836b500700 1 --2- 172.21.15.79:0/314819541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83640cec30 0x7f83640c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.119+0000 7f836b500700 1 --2- 172.21.15.79:0/314819541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f836415a0a0 0x7f83641e5090 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.119+0000 7f836b500700 1 -- 172.21.15.79:0/314819541 >> 172.21.15.79:0/314819541 conn(0x7f8364099cc0 msgr2=0x7f836406ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:53.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.124+0000 7f836b500700 1 -- 172.21.15.79:0/314819541 shutdown_connections 2023-06-23T18:35:53.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.125+0000 7f836b500700 1 -- 172.21.15.79:0/314819541 wait complete. 2023-06-23T18:35:53.133 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data' created 2023-06-23T18:35:53.148 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-23T18:35:53.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.268+0000 7f3fe45a0700 1 Processor -- start 2023-06-23T18:35:53.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.268+0000 7f3fe45a0700 1 -- start start 2023-06-23T18:35:53.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.269+0000 7f3fe45a0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3fdc06a5a0 0x7f3fdc1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:53.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.269+0000 7f3fe45a0700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fdc0cdc90 con 0x7f3fdc1569c0 2023-06-23T18:35:53.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.269+0000 7f3fe45a0700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fdc15d0f0 con 0x7f3fdc0cd440 2023-06-23T18:35:53.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.269+0000 7f3fe45a0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0cf570 con 0x7f3fdc06a5a0 2023-06-23T18:35:53.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.269+0000 7f3fe233c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3fdc1569c0 0x7f3fdc069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42072/0 (socket says 172.21.15.79:42072) 2023-06-23T18:35:53.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.269+0000 7f3fe233c700 1 -- 172.21.15.79:0/3403725173 learned_addr learned my addr 172.21.15.79:0/3403725173 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:53.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.269+0000 7f3fe1b3b700 1 --2- 172.21.15.79:0/3403725173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3fdc06a5a0 0x7f3fdc1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:53.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.269+0000 7f3fe133a700 1 -- 172.21.15.79:0/3403725173 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019484714 0 0) 0x7f3fcc001110 con 0x7f3fdc1569c0 2023-06-23T18:35:53.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.269+0000 7f3fe133a700 1 -- 172.21.15.79:0/3403725173 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc80035b0 con 0x7f3fdc1569c0 2023-06-23T18:35:53.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe133a700 1 -- 172.21.15.79:0/3403725173 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3685187591 0 0) 0x7f3fc80035b0 con 0x7f3fdc1569c0 2023-06-23T18:35:53.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe133a700 1 -- 172.21.15.79:0/3403725173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3fdc06a5a0 msgr2=0x7f3fdc1608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:53.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe133a700 1 --2- 172.21.15.79:0/3403725173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3fdc06a5a0 0x7f3fdc1608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe133a700 1 -- 172.21.15.79:0/3403725173 >> v1:172.21.15.155:6789/0 conn(0x7f3fdc0cd440 legacy=0x7f3fdc0b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe133a700 1 -- 172.21.15.79:0/3403725173 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc156d90 con 0x7f3fdc1569c0 2023-06-23T18:35:53.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe133a700 1 -- 172.21.15.79:0/3403725173 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f3fcc001800 con 0x7f3fdc1569c0 2023-06-23T18:35:53.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe133a700 1 -- 172.21.15.79:0/3403725173 >> v1:172.21.15.79:6789/0 conn(0x7f3fdc1569c0 legacy=0x7f3fdc069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe133a700 1 --2- 172.21.15.79:0/3403725173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fc8003a30 0x7f3fc8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:53.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe133a700 1 --2- 172.21.15.79:0/3403725173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fc80043d0 0x7f3fc8004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:53.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe133a700 1 --2- 172.21.15.79:0/3403725173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3fdc06a5a0 0x7f3fc80050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:53.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe133a700 1 -- 172.21.15.79:0/3403725173 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc800ba10 con 0x7f3fc80043d0 2023-06-23T18:35:53.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe133a700 1 -- 172.21.15.79:0/3403725173 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc800bb50 con 0x7f3fc8003a30 2023-06-23T18:35:53.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe133a700 1 -- 172.21.15.79:0/3403725173 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc800bce0 con 0x7f3fdc06a5a0 2023-06-23T18:35:53.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe1b3b700 1 --2- 172.21.15.79:0/3403725173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fc80043d0 0x7f3fc8004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:53.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe233c700 1 --2- 172.21.15.79:0/3403725173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fc8003a30 0x7f3fc8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:53.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe2b3d700 1 --2- 172.21.15.79:0/3403725173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3fdc06a5a0 0x7f3fc80050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:53.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe1b3b700 1 -- 172.21.15.79:0/3403725173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3fdc06a5a0 msgr2=0x7f3fc80050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:53.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe1b3b700 1 --2- 172.21.15.79:0/3403725173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3fdc06a5a0 0x7f3fc80050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe1b3b700 1 -- 172.21.15.79:0/3403725173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fc8003a30 msgr2=0x7f3fc8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:53.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe1b3b700 1 --2- 172.21.15.79:0/3403725173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fc8003a30 0x7f3fc8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe1b3b700 1 -- 172.21.15.79:0/3403725173 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc80035b0 con 0x7f3fc80043d0 2023-06-23T18:35:53.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.270+0000 7f3fe1b3b700 1 --2- 172.21.15.79:0/3403725173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fc80043d0 0x7f3fc8004890 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009b30 tx=0x7f3fc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a78ecfa0842067 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:53.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.271+0000 7f3fe133a700 1 -- 172.21.15.79:0/3403725173 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd800e6c0 con 0x7f3fc80043d0 2023-06-23T18:35:53.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.271+0000 7f3fe133a700 1 -- 172.21.15.79:0/3403725173 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd800e820 con 0x7f3fc80043d0 2023-06-23T18:35:53.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.271+0000 7f3fe133a700 1 -- 172.21.15.79:0/3403725173 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd800eb90 con 0x7f3fc80043d0 2023-06-23T18:35:53.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.271+0000 7f3fe45a0700 1 -- 172.21.15.79:0/3403725173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fc80043d0 msgr2=0x7f3fc8004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.271+0000 7f3fe45a0700 1 --2- 172.21.15.79:0/3403725173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fc80043d0 0x7f3fc8004890 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009b30 tx=0x7f3fc8005880 comp rx=0 tx=0).stop 2023-06-23T18:35:53.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.271+0000 7f3fe2b3d700 1 -- 172.21.15.79:0/3403725173 reap_dead start 2023-06-23T18:35:53.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.271+0000 7f3fe45a0700 1 -- 172.21.15.79:0/3403725173 shutdown_connections 2023-06-23T18:35:53.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.271+0000 7f3fe45a0700 1 -- 172.21.15.79:0/3403725173 >> 172.21.15.79:0/3403725173 conn(0x7f3fdc06afa0 msgr2=0x7f3fdc06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:53.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.271+0000 7f3fe45a0700 1 -- 172.21.15.79:0/3403725173 shutdown_connections 2023-06-23T18:35:53.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.271+0000 7f3fe45a0700 1 -- 172.21.15.79:0/3403725173 wait complete. 2023-06-23T18:35:53.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.272+0000 7f3fe45a0700 1 Processor -- start 2023-06-23T18:35:53.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.272+0000 7f3fe45a0700 1 -- start start 2023-06-23T18:35:53.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe45a0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fc80043d0 0x7f3fdc152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:53.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe45a0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3fdc0c9880 0x7f3fdc0b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:53.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe45a0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fdc0c4f30 0x7f3fdc0c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:53.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe45a0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0c58e0 con 0x7f3fc80043d0 2023-06-23T18:35:53.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe45a0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0b2ab0 con 0x7f3fdc0c4f30 2023-06-23T18:35:53.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe45a0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc1530e0 con 0x7f3fdc0c9880 2023-06-23T18:35:53.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe233c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fc80043d0 0x7f3fdc152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:53.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe233c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fc80043d0 0x7f3fdc152ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39042/0 (socket says 172.21.15.79:39042) 2023-06-23T18:35:53.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe233c700 1 -- 172.21.15.79:0/2873245498 learned_addr learned my addr 172.21.15.79:0/2873245498 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:53.289 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe2b3d700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fdc0c4f30 0x7f3fdc0c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:53.289 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe1b3b700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3fdc0c9880 0x7f3fdc0b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:53.289 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe233c700 1 -- 172.21.15.79:0/2873245498 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3fdc0c9880 msgr2=0x7f3fdc0b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.289 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe233c700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3fdc0c9880 0x7f3fdc0b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe233c700 1 -- 172.21.15.79:0/2873245498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fdc0c4f30 msgr2=0x7f3fdc0c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe233c700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fdc0c4f30 0x7f3fdc0c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe233c700 1 -- 172.21.15.79:0/2873245498 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc0c69e0 con 0x7f3fc80043d0 2023-06-23T18:35:53.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.273+0000 7f3fe2b3d700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fdc0c4f30 0x7f3fdc0c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:35:53.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.274+0000 7f3fe233c700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fc80043d0 0x7f3fdc152ba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0072c0 tx=0x7f3fcc0075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:53.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.274+0000 7f3fd2ffd700 1 -- 172.21.15.79:0/2873245498 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc007e70 con 0x7f3fc80043d0 2023-06-23T18:35:53.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.274+0000 7f3fd2ffd700 1 -- 172.21.15.79:0/2873245498 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fcc00f6c0 con 0x7f3fc80043d0 2023-06-23T18:35:53.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.274+0000 7f3fe45a0700 1 -- 172.21.15.79:0/2873245498 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc0c6cc0 con 0x7f3fc80043d0 2023-06-23T18:35:53.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.274+0000 7f3fd2ffd700 1 -- 172.21.15.79:0/2873245498 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc00fa50 con 0x7f3fc80043d0 2023-06-23T18:35:53.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.274+0000 7f3fe45a0700 1 -- 172.21.15.79:0/2873245498 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fdc0b5720 con 0x7f3fc80043d0 2023-06-23T18:35:53.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.275+0000 7f3fd2ffd700 1 -- 172.21.15.79:0/2873245498 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3fcc00fbb0 con 0x7f3fc80043d0 2023-06-23T18:35:53.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.275+0000 7f3fe45a0700 1 -- 172.21.15.79:0/2873245498 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fdc0672c0 con 0x7f3fc80043d0 2023-06-23T18:35:53.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.275+0000 7f3fd2ffd700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3fc006ace0 0x7f3fc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:53.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.276+0000 7f3fe1b3b700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3fc006ace0 0x7f3fc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:53.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.276+0000 7f3fd2ffd700 1 -- 172.21.15.79:0/2873245498 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f3fcc088c50 con 0x7f3fc80043d0 2023-06-23T18:35:53.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.276+0000 7f3fe1b3b700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3fc006ace0 0x7f3fc006d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80090b0 tx=0x7f3fd8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:53.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.280+0000 7f3fd2ffd700 1 -- 172.21.15.79:0/2873245498 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3fcc08e020 con 0x7f3fc80043d0 2023-06-23T18:35:53.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.509+0000 7f3fe45a0700 1 -- 172.21.15.79:0/2873245498 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3fdc0cdc90 con 0x7f3fc80043d0 2023-06-23T18:35:53.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.515+0000 7f3fd2ffd700 1 -- 172.21.15.79:0/2873245498 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3fdc0cdc90 con 0x7f3fc80043d0 2023-06-23T18:35:53.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.523+0000 7f3fe45a0700 1 -- 172.21.15.79:0/2873245498 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3fc006ace0 msgr2=0x7f3fc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.523+0000 7f3fe45a0700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3fc006ace0 0x7f3fc006d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80090b0 tx=0x7f3fd8008030 comp rx=0 tx=0).stop 2023-06-23T18:35:53.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.523+0000 7f3fe45a0700 1 -- 172.21.15.79:0/2873245498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fc80043d0 msgr2=0x7f3fdc152ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.523+0000 7f3fe45a0700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fc80043d0 0x7f3fdc152ba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0072c0 tx=0x7f3fcc0075d0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.523+0000 7f3fe45a0700 1 -- 172.21.15.79:0/2873245498 shutdown_connections 2023-06-23T18:35:53.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.523+0000 7f3fe45a0700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3fc006ace0 0x7f3fc006d190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.523+0000 7f3fe45a0700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fdc0c4f30 0x7f3fdc0c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.523+0000 7f3fe45a0700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fc80043d0 0x7f3fdc152ba0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.523+0000 7f3fe45a0700 1 --2- 172.21.15.79:0/2873245498 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3fdc0c9880 0x7f3fdc0b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.523+0000 7f3fe45a0700 1 -- 172.21.15.79:0/2873245498 >> 172.21.15.79:0/2873245498 conn(0x7f3fdc06afa0 msgr2=0x7f3fdc15ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:53.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.524+0000 7f3fe45a0700 1 -- 172.21.15.79:0/2873245498 shutdown_connections 2023-06-23T18:35:53.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.524+0000 7f3fe45a0700 1 -- 172.21.15.79:0/2873245498 wait complete. 2023-06-23T18:35:53.542 INFO:teuthology.orchestra.run.smithi079.stderr:new fs with metadata pool 3 and data pool 4 2023-06-23T18:35:53.550 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:35:53.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.664+0000 7fe63a03c700 1 Processor -- start 2023-06-23T18:35:53.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.664+0000 7fe63a03c700 1 -- start start 2023-06-23T18:35:53.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.665+0000 7fe63a03c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe63415a380 0x7fe6340ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:53.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.665+0000 7fe63a03c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe63415a750 0x7fe6340cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:53.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.665+0000 7fe63a03c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6340b1d40 con 0x7fe6340ade50 2023-06-23T18:35:53.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.665+0000 7fe63a03c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6340b2020 con 0x7fe63415a380 2023-06-23T18:35:53.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.665+0000 7fe63a03c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340ad650 con 0x7fe63415a750 2023-06-23T18:35:53.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.665+0000 7fe6337fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe63415a380 0x7fe6340ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:53.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.665+0000 7fe6337fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe63415a380 0x7fe6340ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39062/0 (socket says 172.21.15.79:39062) 2023-06-23T18:35:53.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.665+0000 7fe6337fe700 1 -- 172.21.15.79:0/1972149956 learned_addr learned my addr 172.21.15.79:0/1972149956 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:53.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.666+0000 7fe6337fe700 1 -- 172.21.15.79:0/1972149956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe63415a750 msgr2=0x7fe6340cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:53.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.666+0000 7fe632ffd700 1 --2- 172.21.15.79:0/1972149956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe63415a750 0x7fe6340cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:53.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.666+0000 7fe6337fe700 1 --2- 172.21.15.79:0/1972149956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe63415a750 0x7fe6340cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.666+0000 7fe6337fe700 1 -- 172.21.15.79:0/1972149956 >> v1:172.21.15.79:6789/0 conn(0x7fe6340ade50 legacy=0x7fe6340ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.666+0000 7fe6327fc700 1 -- 172.21.15.79:0/1972149956 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268579261 0 0) 0x7fe6340b1d40 con 0x7fe6340ade50 2023-06-23T18:35:53.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.666+0000 7fe6337fe700 1 -- 172.21.15.79:0/1972149956 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6340ad930 con 0x7fe63415a380 2023-06-23T18:35:53.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.666+0000 7fe6337fe700 1 --2- 172.21.15.79:0/1972149956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe63415a380 0x7fe6340ce4f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe624006fd0 tx=0x7fe624009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e569ca2173f2b9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:53.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.666+0000 7fe6327fc700 1 -- 172.21.15.79:0/1972149956 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe624005e30 con 0x7fe63415a380 2023-06-23T18:35:53.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.666+0000 7fe6327fc700 1 -- 172.21.15.79:0/1972149956 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe624004030 con 0x7fe63415a380 2023-06-23T18:35:53.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.666+0000 7fe6327fc700 1 -- 172.21.15.79:0/1972149956 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6240043e0 con 0x7fe63415a380 2023-06-23T18:35:53.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.667+0000 7fe63a03c700 1 -- 172.21.15.79:0/1972149956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe63415a380 msgr2=0x7fe6340ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.667+0000 7fe63a03c700 1 --2- 172.21.15.79:0/1972149956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe63415a380 0x7fe6340ce4f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe624006fd0 tx=0x7fe624009bd0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.667+0000 7fe63a03c700 1 -- 172.21.15.79:0/1972149956 shutdown_connections 2023-06-23T18:35:53.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.667+0000 7fe63a03c700 1 --2- 172.21.15.79:0/1972149956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe63415a380 0x7fe6340ce4f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.667+0000 7fe63a03c700 1 --2- 172.21.15.79:0/1972149956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe63415a750 0x7fe6340cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.667+0000 7fe63a03c700 1 -- 172.21.15.79:0/1972149956 >> 172.21.15.79:0/1972149956 conn(0x7fe634089fc0 msgr2=0x7fe634099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:53.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.667+0000 7fe63a03c700 1 -- 172.21.15.79:0/1972149956 shutdown_connections 2023-06-23T18:35:53.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.667+0000 7fe63a03c700 1 -- 172.21.15.79:0/1972149956 wait complete. 2023-06-23T18:35:53.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.668+0000 7fe63a03c700 1 Processor -- start 2023-06-23T18:35:53.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.668+0000 7fe63a03c700 1 -- start start 2023-06-23T18:35:53.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe63a03c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6340ade50 0x7fe63415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:53.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe63a03c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe63415a380 0x7fe634155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:53.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe63a03c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe63415a750 0x7fe6340b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:53.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe63a03c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63415bf10 con 0x7fe6340ade50 2023-06-23T18:35:53.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe63a03c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63415c1f0 con 0x7fe63415a750 2023-06-23T18:35:53.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe63a03c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63415c4d0 con 0x7fe63415a380 2023-06-23T18:35:53.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe6337fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6340ade50 0x7fe63415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:53.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe6337fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6340ade50 0x7fe63415b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39068/0 (socket says 172.21.15.79:39068) 2023-06-23T18:35:53.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe6337fe700 1 -- 172.21.15.79:0/3502681868 learned_addr learned my addr 172.21.15.79:0/3502681868 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:53.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe6337fe700 1 -- 172.21.15.79:0/3502681868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe63415a380 msgr2=0x7fe634155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe6337fe700 1 --2- 172.21.15.79:0/3502681868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe63415a380 0x7fe634155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe6337fe700 1 -- 172.21.15.79:0/3502681868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe63415a750 msgr2=0x7fe6340b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe6337fe700 1 --2- 172.21.15.79:0/3502681868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe63415a750 0x7fe6340b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe6337fe700 1 -- 172.21.15.79:0/3502681868 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe624009500 con 0x7fe6340ade50 2023-06-23T18:35:53.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe6337fe700 1 --2- 172.21.15.79:0/3502681868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6340ade50 0x7fe63415b6c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe624009c10 tx=0x7fe624004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:53.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe61bfff700 1 -- 172.21.15.79:0/3502681868 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe624011620 con 0x7fe6340ade50 2023-06-23T18:35:53.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe61bfff700 1 -- 172.21.15.79:0/3502681868 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe624011780 con 0x7fe6340ade50 2023-06-23T18:35:53.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe63a03c700 1 -- 172.21.15.79:0/3502681868 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63415c7b0 con 0x7fe6340ade50 2023-06-23T18:35:53.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe61bfff700 1 -- 172.21.15.79:0/3502681868 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe624011b30 con 0x7fe6340ade50 2023-06-23T18:35:53.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.669+0000 7fe63a03c700 1 -- 172.21.15.79:0/3502681868 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63415bc00 con 0x7fe6340ade50 2023-06-23T18:35:53.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.670+0000 7fe63a03c700 1 -- 172.21.15.79:0/3502681868 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63415ccd0 con 0x7fe6340ade50 2023-06-23T18:35:53.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.671+0000 7fe61bfff700 1 -- 172.21.15.79:0/3502681868 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe624011c90 con 0x7fe6340ade50 2023-06-23T18:35:53.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.671+0000 7fe61bfff700 1 --2- 172.21.15.79:0/3502681868 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe62006ace0 0x7fe62006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:53.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.671+0000 7fe61bfff700 1 -- 172.21.15.79:0/3502681868 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe62408a7d0 con 0x7fe6340ade50 2023-06-23T18:35:53.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.671+0000 7fe632ffd700 1 --2- 172.21.15.79:0/3502681868 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe62006ace0 0x7fe62006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:53.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.671+0000 7fe632ffd700 1 --2- 172.21.15.79:0/3502681868 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe62006ace0 0x7fe62006d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6340c4bd0 tx=0x7fe628008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:53.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.675+0000 7fe61bfff700 1 -- 172.21.15.79:0/3502681868 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe624055910 con 0x7fe6340ade50 2023-06-23T18:35:53.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.864+0000 7fe63a03c700 1 -- 172.21.15.79:0/3502681868 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe63404e240 con 0x7fe6340ade50 2023-06-23T18:35:53.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.866+0000 7fe61bfff700 1 -- 172.21.15.79:0/3502681868 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+18246 (secure 0 0 0) 0x7fe624058f30 con 0x7fe6340ade50 2023-06-23T18:35:53.871 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:53.872 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":19,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:53.511170+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:35:53.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.868+0000 7fe63a03c700 1 -- 172.21.15.79:0/3502681868 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe62006ace0 msgr2=0x7fe62006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.868+0000 7fe63a03c700 1 --2- 172.21.15.79:0/3502681868 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe62006ace0 0x7fe62006d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6340c4bd0 tx=0x7fe628008040 comp rx=0 tx=0).stop 2023-06-23T18:35:53.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.868+0000 7fe63a03c700 1 -- 172.21.15.79:0/3502681868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6340ade50 msgr2=0x7fe63415b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:53.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.868+0000 7fe63a03c700 1 --2- 172.21.15.79:0/3502681868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6340ade50 0x7fe63415b6c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe624009c10 tx=0x7fe624004750 comp rx=0 tx=0).stop 2023-06-23T18:35:53.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.868+0000 7fe63a03c700 1 -- 172.21.15.79:0/3502681868 shutdown_connections 2023-06-23T18:35:53.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.868+0000 7fe63a03c700 1 --2- 172.21.15.79:0/3502681868 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe62006ace0 0x7fe62006d190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.868+0000 7fe63a03c700 1 --2- 172.21.15.79:0/3502681868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe63415a750 0x7fe6340b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.868+0000 7fe63a03c700 1 --2- 172.21.15.79:0/3502681868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6340ade50 0x7fe63415b6c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.868+0000 7fe63a03c700 1 --2- 172.21.15.79:0/3502681868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe63415a380 0x7fe634155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:53.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.868+0000 7fe63a03c700 1 -- 172.21.15.79:0/3502681868 >> 172.21.15.79:0/3502681868 conn(0x7fe634089fc0 msgr2=0x7fe63406ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:53.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.868+0000 7fe63a03c700 1 -- 172.21.15.79:0/3502681868 shutdown_connections 2023-06-23T18:35:53.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:53.868+0000 7fe63a03c700 1 -- 172.21.15.79:0/3502681868 wait complete. 2023-06-23T18:35:53.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:35:54.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.001+0000 7fdb0ef12700 1 Processor -- start 2023-06-23T18:35:54.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.001+0000 7fdb0ef12700 1 -- start start 2023-06-23T18:35:54.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.002+0000 7fdb0ef12700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdb080b4f20 0x7fdb080c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:54.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.002+0000 7fdb0ef12700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdb080b1f40 0x7fdb08069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:54.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.002+0000 7fdb0ef12700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0806a810 con 0x7fdb0806a3d0 2023-06-23T18:35:54.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.002+0000 7fdb0ef12700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0815a810 con 0x7fdb080b4f20 2023-06-23T18:35:54.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.002+0000 7fdb0ef12700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb080b52f0 con 0x7fdb080b1f40 2023-06-23T18:35:54.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.002+0000 7fdb0ccae700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdb080b4f20 0x7fdb080c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:54.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.002+0000 7fdb0ccae700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdb080b4f20 0x7fdb080c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39076/0 (socket says 172.21.15.79:39076) 2023-06-23T18:35:54.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.002+0000 7fdb0d4af700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fdb0806a3d0 0x7fdb0815df00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42122/0 (socket says 172.21.15.79:42122) 2023-06-23T18:35:54.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.002+0000 7fdb0ccae700 1 -- 172.21.15.79:0/1696415770 learned_addr learned my addr 172.21.15.79:0/1696415770 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:54.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.003+0000 7fdb07fff700 1 --2- 172.21.15.79:0/1696415770 >> v2:172.21.15.155:3300/0 conn(0x7fdb080b1f40 0x7fdb08069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:54.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.003+0000 7fdb0ccae700 1 -- 172.21.15.79:0/1696415770 >> v2:172.21.15.155:3300/0 conn(0x7fdb080b1f40 msgr2=0x7fdb08069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:54.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.003+0000 7fdb0ccae700 1 --2- 172.21.15.79:0/1696415770 >> v2:172.21.15.155:3300/0 conn(0x7fdb080b1f40 0x7fdb08069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.003+0000 7fdb0ccae700 1 -- 172.21.15.79:0/1696415770 >> v1:172.21.15.79:6789/0 conn(0x7fdb0806a3d0 legacy=0x7fdb0815df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:54.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.003+0000 7fdb0ccae700 1 -- 172.21.15.79:0/1696415770 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb08071530 con 0x7fdb080b4f20 2023-06-23T18:35:54.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.003+0000 7fdb077fe700 1 -- 172.21.15.79:0/1696415770 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429885956 0 0) 0x7fdb0806a810 con 0x7fdb0806a3d0 2023-06-23T18:35:54.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.003+0000 7fdb0ccae700 1 --2- 172.21.15.79:0/1696415770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb080b4f20 0x7fdb080c9e40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdb08067190 tx=0x7fdafc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae1ba70a7aad69d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:54.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.003+0000 7fdb077fe700 1 -- 172.21.15.79:0/1696415770 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc00b870 con 0x7fdb080b4f20 2023-06-23T18:35:54.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.003+0000 7fdb077fe700 1 -- 172.21.15.79:0/1696415770 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc00b9d0 con 0x7fdb080b4f20 2023-06-23T18:35:54.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.003+0000 7fdb077fe700 1 -- 172.21.15.79:0/1696415770 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc00bd60 con 0x7fdb080b4f20 2023-06-23T18:35:54.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.004+0000 7fdb0ef12700 1 -- 172.21.15.79:0/1696415770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb080b4f20 msgr2=0x7fdb080c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:54.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.004+0000 7fdb0ef12700 1 --2- 172.21.15.79:0/1696415770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb080b4f20 0x7fdb080c9e40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdb08067190 tx=0x7fdafc009bd0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.004+0000 7fdb0ef12700 1 -- 172.21.15.79:0/1696415770 shutdown_connections 2023-06-23T18:35:54.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.004+0000 7fdb0ef12700 1 --2- 172.21.15.79:0/1696415770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb080b4f20 0x7fdb080c9e40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.004+0000 7fdb0ef12700 1 --2- 172.21.15.79:0/1696415770 >> v2:172.21.15.155:3300/0 conn(0x7fdb080b1f40 0x7fdb08069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.004+0000 7fdb0ef12700 1 -- 172.21.15.79:0/1696415770 >> 172.21.15.79:0/1696415770 conn(0x7fdb0806d660 msgr2=0x7fdb0806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:54.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.004+0000 7fdb0ef12700 1 -- 172.21.15.79:0/1696415770 shutdown_connections 2023-06-23T18:35:54.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.004+0000 7fdb0ef12700 1 -- 172.21.15.79:0/1696415770 wait complete. 2023-06-23T18:35:54.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.005+0000 7fdb0ef12700 1 Processor -- start 2023-06-23T18:35:54.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.005+0000 7fdb0ef12700 1 -- start start 2023-06-23T18:35:54.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.006+0000 7fdb0ef12700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0806a3d0 0x7fdb080c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:54.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.006+0000 7fdb0ef12700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb080b1f40 0x7fdb080c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:54.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.006+0000 7fdb0ef12700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb080b4f20 0x7fdb081495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:54.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.006+0000 7fdb0ef12700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0814d710 con 0x7fdb0806a3d0 2023-06-23T18:35:54.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.006+0000 7fdb0ef12700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0814a810 con 0x7fdb080b4f20 2023-06-23T18:35:54.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.006+0000 7fdb0ef12700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb081f3f70 con 0x7fdb080b1f40 2023-06-23T18:35:54.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.006+0000 7fdb0ccae700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0806a3d0 0x7fdb080c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:54.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.006+0000 7fdb0ccae700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0806a3d0 0x7fdb080c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39082/0 (socket says 172.21.15.79:39082) 2023-06-23T18:35:54.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.006+0000 7fdb0ccae700 1 -- 172.21.15.79:0/2940776575 learned_addr learned my addr 172.21.15.79:0/2940776575 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:54.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.007+0000 7fdb0d4af700 1 --2- 172.21.15.79:0/2940776575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb080b4f20 0x7fdb081495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:54.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.007+0000 7fdb07fff700 1 --2- 172.21.15.79:0/2940776575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb080b1f40 0x7fdb080c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:54.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.007+0000 7fdb0ccae700 1 -- 172.21.15.79:0/2940776575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb080b1f40 msgr2=0x7fdb080c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:54.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.007+0000 7fdb0ccae700 1 --2- 172.21.15.79:0/2940776575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb080b1f40 0x7fdb080c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.007+0000 7fdb0ccae700 1 -- 172.21.15.79:0/2940776575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb080b4f20 msgr2=0x7fdb081495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:54.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.007+0000 7fdb0ccae700 1 --2- 172.21.15.79:0/2940776575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb080b4f20 0x7fdb081495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.007+0000 7fdb0ccae700 1 -- 172.21.15.79:0/2940776575 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc009550 con 0x7fdb0806a3d0 2023-06-23T18:35:54.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.007+0000 7fdb0ccae700 1 --2- 172.21.15.79:0/2940776575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0806a3d0 0x7fdb080c9980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdafc000c00 tx=0x7fdafc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:54.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.007+0000 7fdb057fa700 1 -- 172.21.15.79:0/2940776575 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc004520 con 0x7fdb0806a3d0 2023-06-23T18:35:54.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.007+0000 7fdb057fa700 1 -- 172.21.15.79:0/2940776575 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc01e070 con 0x7fdb0806a3d0 2023-06-23T18:35:54.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.007+0000 7fdb057fa700 1 -- 172.21.15.79:0/2940776575 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0038a0 con 0x7fdb0806a3d0 2023-06-23T18:35:54.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.007+0000 7fdb0ef12700 1 -- 172.21.15.79:0/2940776575 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb081f4250 con 0x7fdb0806a3d0 2023-06-23T18:35:54.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.007+0000 7fdb0ef12700 1 -- 172.21.15.79:0/2940776575 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb081f4810 con 0x7fdb0806a3d0 2023-06-23T18:35:54.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.008+0000 7fdb0ef12700 1 -- 172.21.15.79:0/2940776575 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0804e240 con 0x7fdb0806a3d0 2023-06-23T18:35:54.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.008+0000 7fdb057fa700 1 -- 172.21.15.79:0/2940776575 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdafc003a00 con 0x7fdb0806a3d0 2023-06-23T18:35:54.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.009+0000 7fdb057fa700 1 --2- 172.21.15.79:0/2940776575 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdaec06ace0 0x7fdaec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:54.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.009+0000 7fdb057fa700 1 -- 172.21.15.79:0/2940776575 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdafc020070 con 0x7fdb0806a3d0 2023-06-23T18:35:54.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.009+0000 7fdb07fff700 1 --2- 172.21.15.79:0/2940776575 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdaec06ace0 0x7fdaec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:54.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.009+0000 7fdb07fff700 1 --2- 172.21.15.79:0/2940776575 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdaec06ace0 0x7fdaec06d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdb0814d370 tx=0x7fdaf4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:54.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.013+0000 7fdb057fa700 1 -- 172.21.15.79:0/2940776575 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdafc055b20 con 0x7fdb0806a3d0 2023-06-23T18:35:54.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.204+0000 7fdb0ef12700 1 -- 172.21.15.79:0/2940776575 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb0806e3d0 con 0x7fdb0806a3d0 2023-06-23T18:35:54.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.205+0000 7fdb057fa700 1 -- 172.21.15.79:0/2940776575 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+18246 (secure 0 0 0) 0x7fdafc015090 con 0x7fdb0806a3d0 2023-06-23T18:35:54.210 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:54.210 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":19,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:53.511170+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:35:54.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.207+0000 7fdb0ef12700 1 -- 172.21.15.79:0/2940776575 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdaec06ace0 msgr2=0x7fdaec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:54.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.207+0000 7fdb0ef12700 1 --2- 172.21.15.79:0/2940776575 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdaec06ace0 0x7fdaec06d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdb0814d370 tx=0x7fdaf4008040 comp rx=0 tx=0).stop 2023-06-23T18:35:54.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.207+0000 7fdb0ef12700 1 -- 172.21.15.79:0/2940776575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0806a3d0 msgr2=0x7fdb080c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:54.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.207+0000 7fdb0ef12700 1 --2- 172.21.15.79:0/2940776575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0806a3d0 0x7fdb080c9980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdafc000c00 tx=0x7fdafc004750 comp rx=0 tx=0).stop 2023-06-23T18:35:54.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.207+0000 7fdb0ef12700 1 -- 172.21.15.79:0/2940776575 shutdown_connections 2023-06-23T18:35:54.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.207+0000 7fdb0ef12700 1 --2- 172.21.15.79:0/2940776575 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdaec06ace0 0x7fdaec06d190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.207+0000 7fdb0ef12700 1 --2- 172.21.15.79:0/2940776575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb080b4f20 0x7fdb081495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.207+0000 7fdb0ef12700 1 --2- 172.21.15.79:0/2940776575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb0806a3d0 0x7fdb080c9980 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.207+0000 7fdb0ef12700 1 --2- 172.21.15.79:0/2940776575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb080b1f40 0x7fdb080c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.207+0000 7fdb0ef12700 1 -- 172.21.15.79:0/2940776575 >> 172.21.15.79:0/2940776575 conn(0x7fdb0806d660 msgr2=0x7fdb080b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.207+0000 7fdb0ef12700 1 -- 172.21.15.79:0/2940776575 shutdown_connections 2023-06-23T18:35:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.207+0000 7fdb0ef12700 1 -- 172.21.15.79:0/2940776575 wait complete. 2023-06-23T18:35:54.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-23T18:35:54.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.341+0000 7f5629b9e700 1 Processor -- start 2023-06-23T18:35:54.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.341+0000 7f5629b9e700 1 -- start start 2023-06-23T18:35:54.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.342+0000 7f5629b9e700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f56240cc060 0x7f56240cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:54.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.342+0000 7f5629b9e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5624071710 0x7f5624071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:54.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.342+0000 7f5629b9e700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56240cc9f0 con 0x7f5624156ae0 2023-06-23T18:35:54.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.342+0000 7f5629b9e700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562415a560 con 0x7f56240cc060 2023-06-23T18:35:54.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.342+0000 7f5629b9e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5624159950 con 0x7f5624071710 2023-06-23T18:35:54.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.343+0000 7f5622ffd700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f56240cc060 0x7f56240cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:54.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.343+0000 7f5623fff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5624071710 0x7f5624071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:54.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.343+0000 7f56237fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f5624156ae0 0x7f562415d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:50400/0 (socket says 172.21.15.79:50400) 2023-06-23T18:35:54.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.343+0000 7f56237fe700 1 -- 172.21.15.79:0/3616840621 learned_addr learned my addr 172.21.15.79:0/3616840621 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:54.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.343+0000 7f5623fff700 1 -- 172.21.15.79:0/3616840621 >> v2:172.21.15.155:3300/0 conn(0x7f56240cc060 msgr2=0x7f56240cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:54.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.343+0000 7f5623fff700 1 --2- 172.21.15.79:0/3616840621 >> v2:172.21.15.155:3300/0 conn(0x7f56240cc060 0x7f56240cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.343+0000 7f5623fff700 1 -- 172.21.15.79:0/3616840621 >> v1:172.21.15.155:6789/0 conn(0x7f5624156ae0 legacy=0x7f562415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:54.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.343+0000 7f5623fff700 1 -- 172.21.15.79:0/3616840621 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5624159c30 con 0x7f5624071710 2023-06-23T18:35:54.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.343+0000 7f56227fc700 1 -- 172.21.15.79:0/3616840621 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562629396 0 0) 0x7f5618001110 con 0x7f5624156ae0 2023-06-23T18:35:54.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.343+0000 7f5622ffd700 1 --2- 172.21.15.79:0/3616840621 >> v2:172.21.15.155:3300/0 conn(0x7f56240cc060 0x7f56240cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:54.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.343+0000 7f5623fff700 1 --2- 172.21.15.79:0/3616840621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5624071710 0x7f5624071b90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f561c009d90 tx=0x7f561c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd09f56abd7a879f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:54.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.343+0000 7f56227fc700 1 -- 172.21.15.79:0/3616840621 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c00e070 con 0x7f5624071710 2023-06-23T18:35:54.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.343+0000 7f56227fc700 1 -- 172.21.15.79:0/3616840621 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c00bb20 con 0x7f5624071710 2023-06-23T18:35:54.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.343+0000 7f56227fc700 1 -- 172.21.15.79:0/3616840621 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c012430 con 0x7f5624071710 2023-06-23T18:35:54.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.344+0000 7f5629b9e700 1 -- 172.21.15.79:0/3616840621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5624071710 msgr2=0x7f5624071b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:54.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.344+0000 7f5629b9e700 1 --2- 172.21.15.79:0/3616840621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5624071710 0x7f5624071b90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f561c009d90 tx=0x7f561c00b330 comp rx=0 tx=0).stop 2023-06-23T18:35:54.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.344+0000 7f5629b9e700 1 -- 172.21.15.79:0/3616840621 shutdown_connections 2023-06-23T18:35:54.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.344+0000 7f5629b9e700 1 --2- 172.21.15.79:0/3616840621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5624071710 0x7f5624071b90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.344+0000 7f5629b9e700 1 --2- 172.21.15.79:0/3616840621 >> v2:172.21.15.155:3300/0 conn(0x7f56240cc060 0x7f56240cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.344+0000 7f5629b9e700 1 -- 172.21.15.79:0/3616840621 >> 172.21.15.79:0/3616840621 conn(0x7f562406afa0 msgr2=0x7f562406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:54.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.344+0000 7f5629b9e700 1 -- 172.21.15.79:0/3616840621 shutdown_connections 2023-06-23T18:35:54.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.344+0000 7f5629b9e700 1 -- 172.21.15.79:0/3616840621 wait complete. 2023-06-23T18:35:54.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.345+0000 7f5629b9e700 1 Processor -- start 2023-06-23T18:35:54.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.345+0000 7f5629b9e700 1 -- start start 2023-06-23T18:35:54.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5629b9e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5624071710 0x7f56240c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:54.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5629b9e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56240cc060 0x7f56240c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:54.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5629b9e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5624156ae0 0x7f56240b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:54.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5629b9e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56240c3990 con 0x7f5624156ae0 2023-06-23T18:35:54.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5629b9e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56240c3710 con 0x7f56240cc060 2023-06-23T18:35:54.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5629b9e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56240c6060 con 0x7f5624071710 2023-06-23T18:35:54.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5623fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5624156ae0 0x7f56240b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:54.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5623fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5624156ae0 0x7f56240b46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39100/0 (socket says 172.21.15.79:39100) 2023-06-23T18:35:54.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5623fff700 1 -- 172.21.15.79:0/912056135 learned_addr learned my addr 172.21.15.79:0/912056135 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:54.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5622ffd700 1 --2- 172.21.15.79:0/912056135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56240cc060 0x7f56240c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:54.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f56237fe700 1 --2- 172.21.15.79:0/912056135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5624071710 0x7f56240c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:54.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5623fff700 1 -- 172.21.15.79:0/912056135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5624071710 msgr2=0x7f56240c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:54.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5623fff700 1 --2- 172.21.15.79:0/912056135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5624071710 0x7f56240c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5623fff700 1 -- 172.21.15.79:0/912056135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56240cc060 msgr2=0x7f56240c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:54.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5623fff700 1 --2- 172.21.15.79:0/912056135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56240cc060 0x7f56240c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:54.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.346+0000 7f5623fff700 1 -- 172.21.15.79:0/912056135 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56140094a0 con 0x7f5624156ae0 2023-06-23T18:35:54.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.347+0000 7f5623fff700 1 --2- 172.21.15.79:0/912056135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5624156ae0 0x7f56240b46c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5624159a90 tx=0x7f561c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:54.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.347+0000 7f560bfff700 1 -- 172.21.15.79:0/912056135 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c012430 con 0x7f5624156ae0 2023-06-23T18:35:54.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.347+0000 7f560bfff700 1 -- 172.21.15.79:0/912056135 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c020070 con 0x7f5624156ae0 2023-06-23T18:35:54.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.347+0000 7f5629b9e700 1 -- 172.21.15.79:0/912056135 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561c009a70 con 0x7f5624156ae0 2023-06-23T18:35:54.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.347+0000 7f560bfff700 1 -- 172.21.15.79:0/912056135 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c004280 con 0x7f5624156ae0 2023-06-23T18:35:54.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.347+0000 7f5629b9e700 1 -- 172.21.15.79:0/912056135 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56240b41a0 con 0x7f5624156ae0 2023-06-23T18:35:54.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.348+0000 7f5629b9e700 1 -- 172.21.15.79:0/912056135 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56240672c0 con 0x7f5624156ae0 2023-06-23T18:35:54.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.348+0000 7f560bfff700 1 -- 172.21.15.79:0/912056135 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f561c0043e0 con 0x7f5624156ae0 2023-06-23T18:35:54.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.348+0000 7f560bfff700 1 --2- 172.21.15.79:0/912056135 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f561006ace0 0x7f561006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:54.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.348+0000 7f560bfff700 1 -- 172.21.15.79:0/912056135 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f561c022070 con 0x7f5624156ae0 2023-06-23T18:35:54.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.349+0000 7f56237fe700 1 --2- 172.21.15.79:0/912056135 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f561006ace0 0x7f561006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:54.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.349+0000 7f56237fe700 1 --2- 172.21.15.79:0/912056135 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f561006ace0 0x7f561006d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5618000f60 tx=0x7f5618007e70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:54.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.353+0000 7f560bfff700 1 -- 172.21.15.79:0/912056135 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f561c056500 con 0x7f5624156ae0 2023-06-23T18:35:54.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:54.588+0000 7f5629b9e700 1 -- 172.21.15.79:0/912056135 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f56240b1be0 con 0x7f5624156ae0 2023-06-23T18:35:55.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.528+0000 7f560bfff700 1 -- 172.21.15.79:0/912056135 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f561c059b20 con 0x7f5624156ae0 2023-06-23T18:35:55.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.530+0000 7f5629b9e700 1 -- 172.21.15.79:0/912056135 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f561006ace0 msgr2=0x7f561006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.530+0000 7f5629b9e700 1 --2- 172.21.15.79:0/912056135 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f561006ace0 0x7f561006d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5618000f60 tx=0x7f5618007e70 comp rx=0 tx=0).stop 2023-06-23T18:35:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.530+0000 7f5629b9e700 1 -- 172.21.15.79:0/912056135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5624156ae0 msgr2=0x7f56240b46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.530+0000 7f5629b9e700 1 --2- 172.21.15.79:0/912056135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5624156ae0 0x7f56240b46c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5624159a90 tx=0x7f561c004ad0 comp rx=0 tx=0).stop 2023-06-23T18:35:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.531+0000 7f5629b9e700 1 -- 172.21.15.79:0/912056135 shutdown_connections 2023-06-23T18:35:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.531+0000 7f5629b9e700 1 --2- 172.21.15.79:0/912056135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5624156ae0 0x7f56240b46c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.531+0000 7f5629b9e700 1 --2- 172.21.15.79:0/912056135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5624071710 0x7f56240c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:55.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.531+0000 7f5629b9e700 1 --2- 172.21.15.79:0/912056135 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f561006ace0 0x7f561006d190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:55.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.531+0000 7f5629b9e700 1 --2- 172.21.15.79:0/912056135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56240cc060 0x7f56240c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:55.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.531+0000 7f5629b9e700 1 -- 172.21.15.79:0/912056135 >> 172.21.15.79:0/912056135 conn(0x7f562406afa0 msgr2=0x7f56240b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:55.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.531+0000 7f5629b9e700 1 -- 172.21.15.79:0/912056135 shutdown_connections 2023-06-23T18:35:55.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.531+0000 7f5629b9e700 1 -- 172.21.15.79:0/912056135 wait complete. 2023-06-23T18:35:55.550 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-06-23T18:35:55.550 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-06-23T18:35:55.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.674+0000 7feb5e3e2700 1 Processor -- start 2023-06-23T18:35:55.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.674+0000 7feb5e3e2700 1 -- start start 2023-06-23T18:35:55.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.674+0000 7feb5e3e2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb5815c660 0x7feb580aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:55.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.674+0000 7feb5e3e2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7feb580cdd50 0x7feb580b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:55.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.674+0000 7feb5e3e2700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb580718d0 con 0x7feb580b17d0 2023-06-23T18:35:55.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.674+0000 7feb5e3e2700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb580ca840 con 0x7feb580cdd50 2023-06-23T18:35:55.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.674+0000 7feb5e3e2700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb580b1be0 con 0x7feb5815c660 2023-06-23T18:35:55.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.674+0000 7feb577fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7feb580cdd50 0x7feb580b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:55.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.675+0000 7feb577fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7feb580cdd50 0x7feb580b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39108/0 (socket says 172.21.15.79:39108) 2023-06-23T18:35:55.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.675+0000 7feb577fe700 1 -- 172.21.15.79:0/516927555 learned_addr learned my addr 172.21.15.79:0/516927555 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:55.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.675+0000 7feb57fff700 1 --2- 172.21.15.79:0/516927555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb5815c660 0x7feb580aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:55.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.675+0000 7feb577fe700 1 -- 172.21.15.79:0/516927555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb5815c660 msgr2=0x7feb580aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:55.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.675+0000 7feb577fe700 1 --2- 172.21.15.79:0/516927555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb5815c660 0x7feb580aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:55.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.675+0000 7feb577fe700 1 -- 172.21.15.79:0/516927555 >> v1:172.21.15.155:6789/0 conn(0x7feb580b17d0 legacy=0x7feb5806eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:55.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.675+0000 7feb577fe700 1 -- 172.21.15.79:0/516927555 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5815ca30 con 0x7feb580cdd50 2023-06-23T18:35:55.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.675+0000 7feb577fe700 1 --2- 172.21.15.79:0/516927555 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb580cdd50 0x7feb580b1290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feb480098f0 tx=0x7feb48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33554916fb0e9d9a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:55.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.675+0000 7feb56ffd700 1 -- 172.21.15.79:0/516927555 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4800b8a0 con 0x7feb580cdd50 2023-06-23T18:35:55.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.675+0000 7feb56ffd700 1 -- 172.21.15.79:0/516927555 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4800ba00 con 0x7feb580cdd50 2023-06-23T18:35:55.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.675+0000 7feb56ffd700 1 -- 172.21.15.79:0/516927555 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4800bd90 con 0x7feb580cdd50 2023-06-23T18:35:55.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.676+0000 7feb5e3e2700 1 -- 172.21.15.79:0/516927555 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb580cdd50 msgr2=0x7feb580b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:55.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.676+0000 7feb5e3e2700 1 --2- 172.21.15.79:0/516927555 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb580cdd50 0x7feb580b1290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feb480098f0 tx=0x7feb48009c00 comp rx=0 tx=0).stop 2023-06-23T18:35:55.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.676+0000 7feb5e3e2700 1 -- 172.21.15.79:0/516927555 shutdown_connections 2023-06-23T18:35:55.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.676+0000 7feb5e3e2700 1 --2- 172.21.15.79:0/516927555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb5815c660 0x7feb580aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:55.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.676+0000 7feb5e3e2700 1 --2- 172.21.15.79:0/516927555 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb580cdd50 0x7feb580b1290 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:55.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.676+0000 7feb5e3e2700 1 -- 172.21.15.79:0/516927555 >> 172.21.15.79:0/516927555 conn(0x7feb580cf790 msgr2=0x7feb580cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:55.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.676+0000 7feb5e3e2700 1 -- 172.21.15.79:0/516927555 shutdown_connections 2023-06-23T18:35:55.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.676+0000 7feb5e3e2700 1 -- 172.21.15.79:0/516927555 wait complete. 2023-06-23T18:35:55.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.677+0000 7feb5e3e2700 1 Processor -- start 2023-06-23T18:35:55.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.677+0000 7feb5e3e2700 1 -- start start 2023-06-23T18:35:55.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb5e3e2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb580b17d0 0x7feb580c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:55.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb5e3e2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb580cdd50 0x7feb580af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:55.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb5e3e2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb5815c660 0x7feb580c3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:55.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb5e3e2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5814b490 con 0x7feb580cdd50 2023-06-23T18:35:55.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb5e3e2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb580afa90 con 0x7feb580b17d0 2023-06-23T18:35:55.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb5e3e2700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb580afd70 con 0x7feb5815c660 2023-06-23T18:35:55.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb577fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb580cdd50 0x7feb580af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:55.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb577fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb580cdd50 0x7feb580af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39112/0 (socket says 172.21.15.79:39112) 2023-06-23T18:35:55.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb577fe700 1 -- 172.21.15.79:0/1985126525 learned_addr learned my addr 172.21.15.79:0/1985126525 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:55.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb5c97f700 1 --2- 172.21.15.79:0/1985126525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb5815c660 0x7feb580c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:55.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb57fff700 1 --2- 172.21.15.79:0/1985126525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb580b17d0 0x7feb580c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:55.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb577fe700 1 -- 172.21.15.79:0/1985126525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb5815c660 msgr2=0x7feb580c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:55.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb577fe700 1 --2- 172.21.15.79:0/1985126525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb5815c660 0x7feb580c3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:55.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb577fe700 1 -- 172.21.15.79:0/1985126525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb580b17d0 msgr2=0x7feb580c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:55.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb577fe700 1 --2- 172.21.15.79:0/1985126525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb580b17d0 0x7feb580c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:55.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb577fe700 1 -- 172.21.15.79:0/1985126525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb48009550 con 0x7feb580cdd50 2023-06-23T18:35:55.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb577fe700 1 --2- 172.21.15.79:0/1985126525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb580cdd50 0x7feb580af240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feb48000c00 tx=0x7feb48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:55.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb54ff9700 1 -- 172.21.15.79:0/1985126525 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48004480 con 0x7feb580cdd50 2023-06-23T18:35:55.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.678+0000 7feb54ff9700 1 -- 172.21.15.79:0/1985126525 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4801f070 con 0x7feb580cdd50 2023-06-23T18:35:55.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.679+0000 7feb5e3e2700 1 -- 172.21.15.79:0/1985126525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb580b0050 con 0x7feb580cdd50 2023-06-23T18:35:55.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.679+0000 7feb54ff9700 1 -- 172.21.15.79:0/1985126525 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb480038c0 con 0x7feb580cdd50 2023-06-23T18:35:55.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.679+0000 7feb5e3e2700 1 -- 172.21.15.79:0/1985126525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb580b0610 con 0x7feb580cdd50 2023-06-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.679+0000 7feb5e3e2700 1 -- 172.21.15.79:0/1985126525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb580c9ec0 con 0x7feb580cdd50 2023-06-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.680+0000 7feb54ff9700 1 -- 172.21.15.79:0/1985126525 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feb48021030 con 0x7feb580cdd50 2023-06-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.680+0000 7feb54ff9700 1 --2- 172.21.15.79:0/1985126525 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7feb4406ace0 0x7feb4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.680+0000 7feb54ff9700 1 -- 172.21.15.79:0/1985126525 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feb4808a9c0 con 0x7feb580cdd50 2023-06-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.680+0000 7feb57fff700 1 --2- 172.21.15.79:0/1985126525 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7feb4406ace0 0x7feb4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.681+0000 7feb57fff700 1 --2- 172.21.15.79:0/1985126525 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7feb4406ace0 0x7feb4406d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009930 tx=0x7feb4c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.685+0000 7feb54ff9700 1 -- 172.21.15.79:0/1985126525 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feb48055c20 con 0x7feb580cdd50 2023-06-23T18:35:55.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:55.916+0000 7feb5e3e2700 1 -- 172.21.15.79:0/1985126525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7feb580672c0 con 0x7feb580cdd50 2023-06-23T18:35:56.534 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.530+0000 7feb54ff9700 1 -- 172.21.15.79:0/1985126525 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7feb48059240 con 0x7feb580cdd50 2023-06-23T18:35:56.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.533+0000 7feb5e3e2700 1 -- 172.21.15.79:0/1985126525 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7feb4406ace0 msgr2=0x7feb4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:56.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.533+0000 7feb5e3e2700 1 --2- 172.21.15.79:0/1985126525 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7feb4406ace0 0x7feb4406d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009930 tx=0x7feb4c008040 comp rx=0 tx=0).stop 2023-06-23T18:35:56.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.533+0000 7feb5e3e2700 1 -- 172.21.15.79:0/1985126525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb580cdd50 msgr2=0x7feb580af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:56.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.533+0000 7feb5e3e2700 1 --2- 172.21.15.79:0/1985126525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb580cdd50 0x7feb580af240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feb48000c00 tx=0x7feb48004750 comp rx=0 tx=0).stop 2023-06-23T18:35:56.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.533+0000 7feb5e3e2700 1 -- 172.21.15.79:0/1985126525 shutdown_connections 2023-06-23T18:35:56.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.533+0000 7feb5e3e2700 1 --2- 172.21.15.79:0/1985126525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb5815c660 0x7feb580c3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:56.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.533+0000 7feb5e3e2700 1 --2- 172.21.15.79:0/1985126525 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7feb4406ace0 0x7feb4406d190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:56.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.533+0000 7feb5e3e2700 1 --2- 172.21.15.79:0/1985126525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb580b17d0 0x7feb580c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:56.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.533+0000 7feb5e3e2700 1 --2- 172.21.15.79:0/1985126525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb580cdd50 0x7feb580af240 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:56.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.533+0000 7feb5e3e2700 1 -- 172.21.15.79:0/1985126525 >> 172.21.15.79:0/1985126525 conn(0x7feb580cf790 msgr2=0x7feb58158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:56.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.533+0000 7feb5e3e2700 1 -- 172.21.15.79:0/1985126525 shutdown_connections 2023-06-23T18:35:56.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.533+0000 7feb5e3e2700 1 -- 172.21.15.79:0/1985126525 wait complete. 2023-06-23T18:35:56.552 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-06-23T18:35:56.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:35:56.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.672+0000 7f44f4440700 1 Processor -- start 2023-06-23T18:35:56.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.672+0000 7f44f4440700 1 -- start start 2023-06-23T18:35:56.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.673+0000 7f44f4440700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f44ec0cdad0 0x7f44ec0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:56.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.673+0000 7f44f4440700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44ec0ce620 0x7f44ec0cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:56.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.673+0000 7f44f4440700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ec15cd50 con 0x7f44ec159170 2023-06-23T18:35:56.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.673+0000 7f44f4440700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ec0ceaa0 con 0x7f44ec0cdad0 2023-06-23T18:35:56.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.673+0000 7f44f4440700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec15cfa0 con 0x7f44ec0ce620 2023-06-23T18:35:56.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.673+0000 7f44f19db700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f44ec0cdad0 0x7f44ec0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:56.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.673+0000 7f44f19db700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f44ec0cdad0 0x7f44ec0ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39118/0 (socket says 172.21.15.79:39118) 2023-06-23T18:35:56.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.673+0000 7f44f19db700 1 -- 172.21.15.79:0/1242975841 learned_addr learned my addr 172.21.15.79:0/1242975841 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:56.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.674+0000 7f44f29dd700 1 --2- 172.21.15.79:0/1242975841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44ec0ce620 0x7f44ec0cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:56.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.674+0000 7f44f19db700 1 -- 172.21.15.79:0/1242975841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44ec0ce620 msgr2=0x7f44ec0cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:56.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.674+0000 7f44f19db700 1 --2- 172.21.15.79:0/1242975841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44ec0ce620 0x7f44ec0cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:56.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.674+0000 7f44f19db700 1 -- 172.21.15.79:0/1242975841 >> v1:172.21.15.155:6789/0 conn(0x7f44ec159170 legacy=0x7f44ec0cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:56.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.674+0000 7f44f19db700 1 -- 172.21.15.79:0/1242975841 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec15d280 con 0x7f44ec0cdad0 2023-06-23T18:35:56.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.674+0000 7f44f19db700 1 --2- 172.21.15.79:0/1242975841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44ec0cdad0 0x7f44ec0ce0e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f44e800bb90 tx=0x7f44e800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643b03b82bf7ea3e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:56.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.674+0000 7f44f11da700 1 -- 172.21.15.79:0/1242975841 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e800dd10 con 0x7f44ec0cdad0 2023-06-23T18:35:56.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.674+0000 7f44f11da700 1 -- 172.21.15.79:0/1242975841 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e800de70 con 0x7f44ec0cdad0 2023-06-23T18:35:56.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.674+0000 7f44f11da700 1 -- 172.21.15.79:0/1242975841 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e8004230 con 0x7f44ec0cdad0 2023-06-23T18:35:56.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.675+0000 7f44f4440700 1 -- 172.21.15.79:0/1242975841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44ec0cdad0 msgr2=0x7f44ec0ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:56.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.675+0000 7f44f4440700 1 --2- 172.21.15.79:0/1242975841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44ec0cdad0 0x7f44ec0ce0e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f44e800bb90 tx=0x7f44e800bea0 comp rx=0 tx=0).stop 2023-06-23T18:35:56.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.675+0000 7f44f4440700 1 -- 172.21.15.79:0/1242975841 shutdown_connections 2023-06-23T18:35:56.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.675+0000 7f44f4440700 1 --2- 172.21.15.79:0/1242975841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44ec0ce620 0x7f44ec0cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:56.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.675+0000 7f44f4440700 1 --2- 172.21.15.79:0/1242975841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44ec0cdad0 0x7f44ec0ce0e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:56.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.675+0000 7f44f4440700 1 -- 172.21.15.79:0/1242975841 >> 172.21.15.79:0/1242975841 conn(0x7f44ec06d660 msgr2=0x7f44ec06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:56.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.675+0000 7f44f4440700 1 -- 172.21.15.79:0/1242975841 shutdown_connections 2023-06-23T18:35:56.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.675+0000 7f44f4440700 1 -- 172.21.15.79:0/1242975841 wait complete. 2023-06-23T18:35:56.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.676+0000 7f44f4440700 1 Processor -- start 2023-06-23T18:35:56.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.676+0000 7f44f4440700 1 -- start start 2023-06-23T18:35:56.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f4440700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44ec0cdad0 0x7f44ec0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:56.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f4440700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44ec0ce620 0x7f44ec159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:56.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f4440700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44ec159170 0x7f44ec15a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:56.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f4440700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0ad070 con 0x7f44ec159170 2023-06-23T18:35:56.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f4440700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0ad350 con 0x7f44ec0ce620 2023-06-23T18:35:56.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f4440700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0ad630 con 0x7f44ec0cdad0 2023-06-23T18:35:56.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f29dd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44ec159170 0x7f44ec15a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:56.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f29dd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44ec159170 0x7f44ec15a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39126/0 (socket says 172.21.15.79:39126) 2023-06-23T18:35:56.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f29dd700 1 -- 172.21.15.79:0/2664968980 learned_addr learned my addr 172.21.15.79:0/2664968980 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:56.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f19db700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44ec0ce620 0x7f44ec159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:56.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f21dc700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44ec0cdad0 0x7f44ec0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:56.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f29dd700 1 -- 172.21.15.79:0/2664968980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44ec0cdad0 msgr2=0x7f44ec0b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:56.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f29dd700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44ec0cdad0 0x7f44ec0b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:56.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f29dd700 1 -- 172.21.15.79:0/2664968980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44ec0ce620 msgr2=0x7f44ec159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:56.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f29dd700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44ec0ce620 0x7f44ec159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:56.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f29dd700 1 -- 172.21.15.79:0/2664968980 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e800b7f0 con 0x7f44ec159170 2023-06-23T18:35:56.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f21dc700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44ec0cdad0 0x7f44ec0b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:35:56.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f29dd700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44ec159170 0x7f44ec15a460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f44e400cd70 tx=0x7f44e400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:56.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44e2ffd700 1 -- 172.21.15.79:0/2664968980 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4003a20 con 0x7f44ec159170 2023-06-23T18:35:56.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44e2ffd700 1 -- 172.21.15.79:0/2664968980 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e4003b80 con 0x7f44ec159170 2023-06-23T18:35:56.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f4440700 1 -- 172.21.15.79:0/2664968980 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ec0ad910 con 0x7f44ec159170 2023-06-23T18:35:56.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44e2ffd700 1 -- 172.21.15.79:0/2664968980 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4004020 con 0x7f44ec159170 2023-06-23T18:35:56.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.677+0000 7f44f4440700 1 -- 172.21.15.79:0/2664968980 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ec0b53c0 con 0x7f44ec159170 2023-06-23T18:35:56.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.678+0000 7f44f4440700 1 -- 172.21.15.79:0/2664968980 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ec0ade30 con 0x7f44ec159170 2023-06-23T18:35:56.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.679+0000 7f44e2ffd700 1 -- 172.21.15.79:0/2664968980 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f44e4004180 con 0x7f44ec159170 2023-06-23T18:35:56.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.679+0000 7f44e2ffd700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f44d006ace0 0x7f44d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:56.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.679+0000 7f44e2ffd700 1 -- 172.21.15.79:0/2664968980 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f44e4012070 con 0x7f44ec159170 2023-06-23T18:35:56.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.679+0000 7f44f21dc700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f44d006ace0 0x7f44d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:56.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.680+0000 7f44f21dc700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f44d006ace0 0x7f44d006d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f44ec15c850 tx=0x7f44dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:56.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.683+0000 7f44e2ffd700 1 -- 172.21.15.79:0/2664968980 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f44ec0ade30 con 0x7f44ec159170 2023-06-23T18:35:56.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.913+0000 7f44f4440700 1 -- 172.21.15.79:0/2664968980 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44ec0ade30 con 0x7f44ec159170 2023-06-23T18:35:56.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.914+0000 7f44e2ffd700 1 -- 172.21.15.79:0/2664968980 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f44ec0ade30 con 0x7f44ec159170 2023-06-23T18:35:56.919 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:56.919 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/399719238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":399719238},{"type":"v1","addr":"172.21.15.155:6834","nonce":399719238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1846440916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1846440916},{"type":"v1","addr":"172.21.15.155:6835","nonce":1846440916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3310180303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3310180303},{"type":"v1","addr":"172.21.15.79:6835","nonce":3310180303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:35:53.511495+0000","modified":"2023-06-23T18:35:56.527609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6837/3900137670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3900137670},{"type":"v1","addr":"172.21.15.79:6837","nonce":3900137670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:35:56.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.916+0000 7f44f4440700 1 -- 172.21.15.79:0/2664968980 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f44d006ace0 msgr2=0x7f44d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:56.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.916+0000 7f44f4440700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f44d006ace0 0x7f44d006d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f44ec15c850 tx=0x7f44dc008040 comp rx=0 tx=0).stop 2023-06-23T18:35:56.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.916+0000 7f44f4440700 1 -- 172.21.15.79:0/2664968980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44ec159170 msgr2=0x7f44ec15a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:56.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.916+0000 7f44f4440700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44ec159170 0x7f44ec15a460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f44e400cd70 tx=0x7f44e400a570 comp rx=0 tx=0).stop 2023-06-23T18:35:56.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.916+0000 7f44f4440700 1 -- 172.21.15.79:0/2664968980 shutdown_connections 2023-06-23T18:35:56.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.916+0000 7f44f4440700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44ec159170 0x7f44ec15a460 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:56.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.916+0000 7f44f4440700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44ec0cdad0 0x7f44ec0b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:56.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.916+0000 7f44f4440700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f44d006ace0 0x7f44d006d190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:56.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.916+0000 7f44f4440700 1 --2- 172.21.15.79:0/2664968980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44ec0ce620 0x7f44ec159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:56.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.916+0000 7f44f4440700 1 -- 172.21.15.79:0/2664968980 >> 172.21.15.79:0/2664968980 conn(0x7f44ec06d660 msgr2=0x7f44ec071640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:56.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.916+0000 7f44f4440700 1 -- 172.21.15.79:0/2664968980 shutdown_connections 2023-06-23T18:35:56.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:56.916+0000 7f44f4440700 1 -- 172.21.15.79:0/2664968980 wait complete. 2023-06-23T18:35:56.923 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 8 2023-06-23T18:35:56.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:35:57.053 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.048+0000 7f25c95cb700 1 Processor -- start 2023-06-23T18:35:57.054 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.048+0000 7f25c95cb700 1 -- start start 2023-06-23T18:35:57.061 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c95cb700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c40d3710 0x7f25c40cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.061 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c95cb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f25c40cfa90 0x7f25c40d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.061 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c95cb700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c40cd910 con 0x7f25c40b83c0 2023-06-23T18:35:57.062 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c95cb700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c40cfee0 con 0x7f25c40cfa90 2023-06-23T18:35:57.062 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c95cb700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c40d0a80 con 0x7f25c40d3710 2023-06-23T18:35:57.062 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c37fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f25c40cfa90 0x7f25c40d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c2ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f25c40b83c0 0x7f25c40cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42172/0 (socket says 172.21.15.79:42172) 2023-06-23T18:35:57.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c2ffd700 1 -- 172.21.15.79:0/3663252108 learned_addr learned my addr 172.21.15.79:0/3663252108 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:57.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c27fc700 1 --2- 172.21.15.79:0/3663252108 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c40d3710 0x7f25c40cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c37fe700 1 -- 172.21.15.79:0/3663252108 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c40d3710 msgr2=0x7f25c40cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:57.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c37fe700 1 --2- 172.21.15.79:0/3663252108 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c40d3710 0x7f25c40cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c37fe700 1 -- 172.21.15.79:0/3663252108 >> v1:172.21.15.79:6789/0 conn(0x7f25c40b83c0 legacy=0x7f25c40cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c37fe700 1 -- 172.21.15.79:0/3663252108 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c40b8790 con 0x7f25c40cfa90 2023-06-23T18:35:57.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c1ffb700 1 -- 172.21.15.79:0/3663252108 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070492599 0 0) 0x7f25ac001110 con 0x7f25c40b83c0 2023-06-23T18:35:57.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.049+0000 7f25c37fe700 1 --2- 172.21.15.79:0/3663252108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c40cfa90 0x7f25c40d0f10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009d90 tx=0x7f25bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab1d0759514bc46 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:57.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.050+0000 7f25c1ffb700 1 -- 172.21.15.79:0/3663252108 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc00e070 con 0x7f25c40cfa90 2023-06-23T18:35:57.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.050+0000 7f25c1ffb700 1 -- 172.21.15.79:0/3663252108 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc00bbd0 con 0x7f25c40cfa90 2023-06-23T18:35:57.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.050+0000 7f25c1ffb700 1 -- 172.21.15.79:0/3663252108 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc012430 con 0x7f25c40cfa90 2023-06-23T18:35:57.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.050+0000 7f25c95cb700 1 -- 172.21.15.79:0/3663252108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c40cfa90 msgr2=0x7f25c40d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.050+0000 7f25c95cb700 1 --2- 172.21.15.79:0/3663252108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c40cfa90 0x7f25c40d0f10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009d90 tx=0x7f25bc00b330 comp rx=0 tx=0).stop 2023-06-23T18:35:57.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.050+0000 7f25c95cb700 1 -- 172.21.15.79:0/3663252108 shutdown_connections 2023-06-23T18:35:57.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.050+0000 7f25c95cb700 1 --2- 172.21.15.79:0/3663252108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c40cfa90 0x7f25c40d0f10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.050+0000 7f25c95cb700 1 --2- 172.21.15.79:0/3663252108 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c40d3710 0x7f25c40cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.050+0000 7f25c95cb700 1 -- 172.21.15.79:0/3663252108 >> 172.21.15.79:0/3663252108 conn(0x7f25c406b350 msgr2=0x7f25c406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:57.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.050+0000 7f25c95cb700 1 -- 172.21.15.79:0/3663252108 shutdown_connections 2023-06-23T18:35:57.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.050+0000 7f25c95cb700 1 -- 172.21.15.79:0/3663252108 wait complete. 2023-06-23T18:35:57.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.051+0000 7f25c95cb700 1 Processor -- start 2023-06-23T18:35:57.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.052+0000 7f25c95cb700 1 -- start start 2023-06-23T18:35:57.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.052+0000 7f25c95cb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c40b83c0 0x7f25c40c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.052+0000 7f25c95cb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c40cfa90 0x7f25c415f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.052+0000 7f25c95cb700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c40d3710 0x7f25c4149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.052+0000 7f25c95cb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c415de10 con 0x7f25c40cfa90 2023-06-23T18:35:57.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.052+0000 7f25c95cb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c415e7a0 con 0x7f25c40b83c0 2023-06-23T18:35:57.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.052+0000 7f25c95cb700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c415e060 con 0x7f25c40d3710 2023-06-23T18:35:57.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.052+0000 7f25c27fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c40cfa90 0x7f25c415f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.052+0000 7f25c27fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c40cfa90 0x7f25c415f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39136/0 (socket says 172.21.15.79:39136) 2023-06-23T18:35:57.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.052+0000 7f25c27fc700 1 -- 172.21.15.79:0/172139482 learned_addr learned my addr 172.21.15.79:0/172139482 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:57.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.053+0000 7f25c37fe700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c40d3710 0x7f25c4149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.053+0000 7f25c2ffd700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c40b83c0 0x7f25c40c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.053+0000 7f25c27fc700 1 -- 172.21.15.79:0/172139482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c40d3710 msgr2=0x7f25c4149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.053+0000 7f25c27fc700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c40d3710 0x7f25c4149d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.053+0000 7f25c27fc700 1 -- 172.21.15.79:0/172139482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c40b83c0 msgr2=0x7f25c40c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.053+0000 7f25c27fc700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c40b83c0 0x7f25c40c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.053+0000 7f25c27fc700 1 -- 172.21.15.79:0/172139482 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc009a70 con 0x7f25c40cfa90 2023-06-23T18:35:57.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.053+0000 7f25c2ffd700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c40b83c0 0x7f25c40c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:35:57.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.053+0000 7f25c27fc700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c40cfa90 0x7f25c415f1f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f25b400d800 tx=0x7f25b400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:57.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.053+0000 7f25bb7fe700 1 -- 172.21.15.79:0/172139482 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b40125f0 con 0x7f25c40cfa90 2023-06-23T18:35:57.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.053+0000 7f25bb7fe700 1 -- 172.21.15.79:0/172139482 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b4012750 con 0x7f25c40cfa90 2023-06-23T18:35:57.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.053+0000 7f25c95cb700 1 -- 172.21.15.79:0/172139482 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c415e340 con 0x7f25c40cfa90 2023-06-23T18:35:57.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.053+0000 7f25bb7fe700 1 -- 172.21.15.79:0/172139482 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b4012ae0 con 0x7f25c40cfa90 2023-06-23T18:35:57.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.053+0000 7f25c95cb700 1 -- 172.21.15.79:0/172139482 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c415ecd0 con 0x7f25c40cfa90 2023-06-23T18:35:57.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.054+0000 7f25c95cb700 1 -- 172.21.15.79:0/172139482 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c415ee60 con 0x7f25c40cfa90 2023-06-23T18:35:57.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.054+0000 7f25bb7fe700 1 -- 172.21.15.79:0/172139482 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f25b4012c40 con 0x7f25c40cfa90 2023-06-23T18:35:57.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.055+0000 7f25bb7fe700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f25a406ace0 0x7f25a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.055+0000 7f25bb7fe700 1 -- 172.21.15.79:0/172139482 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f25b4089810 con 0x7f25c40cfa90 2023-06-23T18:35:57.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.055+0000 7f25c2ffd700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f25a406ace0 0x7f25a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.055+0000 7f25c2ffd700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f25a406ace0 0x7f25a406d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f25ac007fd0 tx=0x7f25ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:57.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.059+0000 7f25bb7fe700 1 -- 172.21.15.79:0/172139482 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f25b40548c0 con 0x7f25c40cfa90 2023-06-23T18:35:57.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.247+0000 7f25c95cb700 1 -- 172.21.15.79:0/172139482 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25c415ac90 con 0x7f25c40cfa90 2023-06-23T18:35:57.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.248+0000 7f25bb7fe700 1 -- 172.21.15.79:0/172139482 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7f25b4057ee0 con 0x7f25c40cfa90 2023-06-23T18:35:57.253 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:57.253 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:35:57.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.250+0000 7f25c95cb700 1 -- 172.21.15.79:0/172139482 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f25a406ace0 msgr2=0x7f25a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.250+0000 7f25c95cb700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f25a406ace0 0x7f25a406d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f25ac007fd0 tx=0x7f25ac009040 comp rx=0 tx=0).stop 2023-06-23T18:35:57.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.250+0000 7f25c95cb700 1 -- 172.21.15.79:0/172139482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c40cfa90 msgr2=0x7f25c415f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.250+0000 7f25c95cb700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c40cfa90 0x7f25c415f1f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f25b400d800 tx=0x7f25b400dbc0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.250+0000 7f25c95cb700 1 -- 172.21.15.79:0/172139482 shutdown_connections 2023-06-23T18:35:57.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.250+0000 7f25c95cb700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c40d3710 0x7f25c4149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.250+0000 7f25c95cb700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f25a406ace0 0x7f25a406d190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.250+0000 7f25c95cb700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c40b83c0 0x7f25c40c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.250+0000 7f25c95cb700 1 --2- 172.21.15.79:0/172139482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c40cfa90 0x7f25c415f1f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.250+0000 7f25c95cb700 1 -- 172.21.15.79:0/172139482 >> 172.21.15.79:0/172139482 conn(0x7f25c406b350 msgr2=0x7f25c40b6920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:57.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.250+0000 7f25c95cb700 1 -- 172.21.15.79:0/172139482 shutdown_connections 2023-06-23T18:35:57.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.250+0000 7f25c95cb700 1 -- 172.21.15.79:0/172139482 wait complete. 2023-06-23T18:35:57.267 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-06-23T18:35:57.267 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:35:57.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.382+0000 7fc0ef9ca700 1 Processor -- start 2023-06-23T18:35:57.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.382+0000 7fc0ef9ca700 1 -- start start 2023-06-23T18:35:57.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ef9ca700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc0e8071490 0x7fc0e8071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ef9ca700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc0e80b1270 0x7fc0e8071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ef9ca700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e80aee30 0x7fc0e8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ef9ca700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e81608f0 con 0x7fc0e80b1270 2023-06-23T18:35:57.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ef9ca700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e80722e0 con 0x7fc0e8071490 2023-06-23T18:35:57.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ef9ca700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e8160e20 con 0x7fc0e80aee30 2023-06-23T18:35:57.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ecf65700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc0e80b1270 0x7fc0e8071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ecf65700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc0e80b1270 0x7fc0e8071da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39146/0 (socket says 172.21.15.79:39146) 2023-06-23T18:35:57.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ecf65700 1 -- 172.21.15.79:0/3784177736 learned_addr learned my addr 172.21.15.79:0/3784177736 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:57.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0edf67700 1 --2- 172.21.15.79:0/3784177736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e80aee30 0x7fc0e8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ed766700 1 --2- 172.21.15.79:0/3784177736 >> v2:172.21.15.155:3300/0 conn(0x7fc0e8071490 0x7fc0e8071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ecf65700 1 -- 172.21.15.79:0/3784177736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e80aee30 msgr2=0x7fc0e8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:57.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ecf65700 1 --2- 172.21.15.79:0/3784177736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e80aee30 0x7fc0e8069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ecf65700 1 -- 172.21.15.79:0/3784177736 >> v2:172.21.15.155:3300/0 conn(0x7fc0e8071490 msgr2=0x7fc0e8071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:57.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ecf65700 1 --2- 172.21.15.79:0/3784177736 >> v2:172.21.15.155:3300/0 conn(0x7fc0e8071490 0x7fc0e8071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.383+0000 7fc0ecf65700 1 -- 172.21.15.79:0/3784177736 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e80cd4f0 con 0x7fc0e80b1270 2023-06-23T18:35:57.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.384+0000 7fc0ecf65700 1 --2- 172.21.15.79:0/3784177736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0e80b1270 0x7fc0e8071da0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8070f40 tx=0x7fc0e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53e15cce91d9f7b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:57.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.384+0000 7fc0dffff700 1 -- 172.21.15.79:0/3784177736 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4011830 con 0x7fc0e80b1270 2023-06-23T18:35:57.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.384+0000 7fc0dffff700 1 -- 172.21.15.79:0/3784177736 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e4011990 con 0x7fc0e80b1270 2023-06-23T18:35:57.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.384+0000 7fc0dffff700 1 -- 172.21.15.79:0/3784177736 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4011d20 con 0x7fc0e80b1270 2023-06-23T18:35:57.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.384+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/3784177736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0e80b1270 msgr2=0x7fc0e8071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.384+0000 7fc0ef9ca700 1 --2- 172.21.15.79:0/3784177736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0e80b1270 0x7fc0e8071da0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8070f40 tx=0x7fc0e4009bd0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.384+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/3784177736 shutdown_connections 2023-06-23T18:35:57.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.384+0000 7fc0ef9ca700 1 --2- 172.21.15.79:0/3784177736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e80aee30 0x7fc0e8069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.384+0000 7fc0ef9ca700 1 --2- 172.21.15.79:0/3784177736 >> v2:172.21.15.155:3300/0 conn(0x7fc0e8071490 0x7fc0e8071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.384+0000 7fc0ef9ca700 1 --2- 172.21.15.79:0/3784177736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0e80b1270 0x7fc0e8071da0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.384+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/3784177736 >> 172.21.15.79:0/3784177736 conn(0x7fc0e806d660 msgr2=0x7fc0e806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:57.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.384+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/3784177736 shutdown_connections 2023-06-23T18:35:57.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.384+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/3784177736 wait complete. 2023-06-23T18:35:57.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.386+0000 7fc0ef9ca700 1 Processor -- start 2023-06-23T18:35:57.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.386+0000 7fc0ef9ca700 1 -- start start 2023-06-23T18:35:57.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.386+0000 7fc0ef9ca700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0e8071490 0x7fc0e80cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.386+0000 7fc0ef9ca700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e80aee30 0x7fc0e80b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.386+0000 7fc0ef9ca700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e80b1270 0x7fc0e81510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.386+0000 7fc0ef9ca700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e80c60e0 con 0x7fc0e8071490 2023-06-23T18:35:57.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.386+0000 7fc0ef9ca700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e80caab0 con 0x7fc0e80aee30 2023-06-23T18:35:57.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.386+0000 7fc0ef9ca700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e80cad90 con 0x7fc0e80b1270 2023-06-23T18:35:57.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.386+0000 7fc0ed766700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0e8071490 0x7fc0e80cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.386+0000 7fc0ed766700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0e8071490 0x7fc0e80cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39150/0 (socket says 172.21.15.79:39150) 2023-06-23T18:35:57.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.386+0000 7fc0ed766700 1 -- 172.21.15.79:0/2548494913 learned_addr learned my addr 172.21.15.79:0/2548494913 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:57.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.387+0000 7fc0ecf65700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e80aee30 0x7fc0e80b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.387+0000 7fc0edf67700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e80b1270 0x7fc0e81510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.387+0000 7fc0ed766700 1 -- 172.21.15.79:0/2548494913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e80b1270 msgr2=0x7fc0e81510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.387+0000 7fc0ed766700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e80b1270 0x7fc0e81510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.387+0000 7fc0ed766700 1 -- 172.21.15.79:0/2548494913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e80aee30 msgr2=0x7fc0e80b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.387+0000 7fc0ed766700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e80aee30 0x7fc0e80b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.387+0000 7fc0ed766700 1 -- 172.21.15.79:0/2548494913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e4009550 con 0x7fc0e8071490 2023-06-23T18:35:57.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.387+0000 7fc0edf67700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e80b1270 0x7fc0e81510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:35:57.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.387+0000 7fc0ed766700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0e8071490 0x7fc0e80cc8b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800e870 tx=0x7fc0d800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:57.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.387+0000 7fc0ddffb700 1 -- 172.21.15.79:0/2548494913 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8009590 con 0x7fc0e8071490 2023-06-23T18:35:57.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.387+0000 7fc0ddffb700 1 -- 172.21.15.79:0/2548494913 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d8013680 con 0x7fc0e8071490 2023-06-23T18:35:57.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.387+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/2548494913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e80cb070 con 0x7fc0e8071490 2023-06-23T18:35:57.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.387+0000 7fc0ddffb700 1 -- 172.21.15.79:0/2548494913 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d80139f0 con 0x7fc0e8071490 2023-06-23T18:35:57.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.387+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/2548494913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e8151600 con 0x7fc0e8071490 2023-06-23T18:35:57.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.388+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/2548494913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e80cb6c0 con 0x7fc0e8071490 2023-06-23T18:35:57.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.389+0000 7fc0ddffb700 1 -- 172.21.15.79:0/2548494913 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc0d8013b50 con 0x7fc0e8071490 2023-06-23T18:35:57.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.389+0000 7fc0ddffb700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc0cc06ace0 0x7fc0cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.389+0000 7fc0ddffb700 1 -- 172.21.15.79:0/2548494913 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc0d80892e0 con 0x7fc0e8071490 2023-06-23T18:35:57.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.389+0000 7fc0ecf65700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc0cc06ace0 0x7fc0cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.390+0000 7fc0ecf65700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc0cc06ace0 0x7fc0cc06d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8070f40 tx=0x7fc0e4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:57.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.394+0000 7fc0ddffb700 1 -- 172.21.15.79:0/2548494913 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc0d8054420 con 0x7fc0e8071490 2023-06-23T18:35:57.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.578+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/2548494913 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0e80cd790 con 0x7fc0cc06ace0 2023-06-23T18:35:57.591 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.586+0000 7fc0ddffb700 1 -- 172.21.15.79:0/2548494913 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287007 (secure 0 0 0) 0x7fc0e4004040 con 0x7fc0cc06ace0 2023-06-23T18:35:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:57.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.590+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/2548494913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc0cc06ace0 msgr2=0x7fc0cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.590+0000 7fc0ef9ca700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc0cc06ace0 0x7fc0cc06d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8070f40 tx=0x7fc0e4008030 comp rx=0 tx=0).stop 2023-06-23T18:35:57.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.590+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/2548494913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0e8071490 msgr2=0x7fc0e80cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.590+0000 7fc0ef9ca700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0e8071490 0x7fc0e80cc8b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800e870 tx=0x7fc0d800eb80 comp rx=0 tx=0).stop 2023-06-23T18:35:57.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.590+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/2548494913 shutdown_connections 2023-06-23T18:35:57.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.590+0000 7fc0ef9ca700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e80b1270 0x7fc0e81510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.590+0000 7fc0ef9ca700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0e8071490 0x7fc0e80cc8b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.590+0000 7fc0ef9ca700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc0cc06ace0 0x7fc0cc06d190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.590+0000 7fc0ef9ca700 1 --2- 172.21.15.79:0/2548494913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e80aee30 0x7fc0e80b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.590+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/2548494913 >> 172.21.15.79:0/2548494913 conn(0x7fc0e806d660 msgr2=0x7fc0e80cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:57.598 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.590+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/2548494913 shutdown_connections 2023-06-23T18:35:57.598 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.590+0000 7fc0ef9ca700 1 -- 172.21.15.79:0/2548494913 wait complete. 2023-06-23T18:35:57.598 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-23T18:35:57.606 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-06-23T18:35:56.695869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218416,"kb_used_data":2800,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756304,"statfs":{"total":773094113280,"available":772870455296,"internally_reserved":0,"allocated":2867200,"data_stored":1770148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.586464"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287219+0000","last_change":"2023-06-23T18:35:53.534999+0000","last_active":"2023-06-23T18:35:55.287219+0000","last_peered":"2023-06-23T18:35:55.287219+0000","last_clean":"2023-06-23T18:35:55.287219+0000","last_became_active":"2023-06-23T18:35:53.534775+0000","last_became_peered":"2023-06-23T18:35:53.534775+0000","last_unstale":"2023-06-23T18:35:55.287219+0000","last_undegraded":"2023-06-23T18:35:55.287219+0000","last_fullsized":"2023-06-23T18:35:55.287219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:15:42.949792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527064+0000","last_change":"2023-06-23T18:35:53.537237+0000","last_active":"2023-06-23T18:35:54.527064+0000","last_peered":"2023-06-23T18:35:54.527064+0000","last_clean":"2023-06-23T18:35:54.527064+0000","last_became_active":"2023-06-23T18:35:53.536900+0000","last_became_peered":"2023-06-23T18:35:53.536900+0000","last_unstale":"2023-06-23T18:35:54.527064+0000","last_undegraded":"2023-06-23T18:35:54.527064+0000","last_fullsized":"2023-06-23T18:35:54.527064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:17.520121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526243+0000","last_change":"2023-06-23T18:35:53.527037+0000","last_active":"2023-06-23T18:35:54.526243+0000","last_peered":"2023-06-23T18:35:54.526243+0000","last_clean":"2023-06-23T18:35:54.526243+0000","last_became_active":"2023-06-23T18:35:53.526655+0000","last_became_peered":"2023-06-23T18:35:53.526655+0000","last_unstale":"2023-06-23T18:35:54.526243+0000","last_undegraded":"2023-06-23T18:35:54.526243+0000","last_fullsized":"2023-06-23T18:35:54.526243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:30.349972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257396+0000","last_change":"2023-06-23T18:35:53.540867+0000","last_active":"2023-06-23T18:35:55.257396+0000","last_peered":"2023-06-23T18:35:55.257396+0000","last_clean":"2023-06-23T18:35:55.257396+0000","last_became_active":"2023-06-23T18:35:53.540572+0000","last_became_peered":"2023-06-23T18:35:53.540572+0000","last_unstale":"2023-06-23T18:35:55.257396+0000","last_undegraded":"2023-06-23T18:35:55.257396+0000","last_fullsized":"2023-06-23T18:35:55.257396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:28.977806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287682+0000","last_change":"2023-06-23T18:35:53.535027+0000","last_active":"2023-06-23T18:35:55.287682+0000","last_peered":"2023-06-23T18:35:55.287682+0000","last_clean":"2023-06-23T18:35:55.287682+0000","last_became_active":"2023-06-23T18:35:53.534793+0000","last_became_peered":"2023-06-23T18:35:53.534793+0000","last_unstale":"2023-06-23T18:35:55.287682+0000","last_undegraded":"2023-06-23T18:35:55.287682+0000","last_fullsized":"2023-06-23T18:35:55.287682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:17.016870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526443+0000","last_change":"2023-06-23T18:35:53.524551+0000","last_active":"2023-06-23T18:35:54.526443+0000","last_peered":"2023-06-23T18:35:54.526443+0000","last_clean":"2023-06-23T18:35:54.526443+0000","last_became_active":"2023-06-23T18:35:53.524271+0000","last_became_peered":"2023-06-23T18:35:53.524271+0000","last_unstale":"2023-06-23T18:35:54.526443+0000","last_undegraded":"2023-06-23T18:35:54.526443+0000","last_fullsized":"2023-06-23T18:35:54.526443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:08:04.256786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793931+0000","last_change":"2023-06-23T18:35:53.540542+0000","last_active":"2023-06-23T18:35:54.793931+0000","last_peered":"2023-06-23T18:35:54.793931+0000","last_clean":"2023-06-23T18:35:54.793931+0000","last_became_active":"2023-06-23T18:35:53.539978+0000","last_became_peered":"2023-06-23T18:35:53.539978+0000","last_unstale":"2023-06-23T18:35:54.793931+0000","last_undegraded":"2023-06-23T18:35:54.793931+0000","last_fullsized":"2023-06-23T18:35:54.793931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:57:46.218466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525592+0000","last_change":"2023-06-23T18:35:53.533541+0000","last_active":"2023-06-23T18:35:54.525592+0000","last_peered":"2023-06-23T18:35:54.525592+0000","last_clean":"2023-06-23T18:35:54.525592+0000","last_became_active":"2023-06-23T18:35:53.531085+0000","last_became_peered":"2023-06-23T18:35:53.531085+0000","last_unstale":"2023-06-23T18:35:54.525592+0000","last_undegraded":"2023-06-23T18:35:54.525592+0000","last_fullsized":"2023-06-23T18:35:54.525592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:52.658179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288165+0000","last_change":"2023-06-23T18:35:53.534716+0000","last_active":"2023-06-23T18:35:55.288165+0000","last_peered":"2023-06-23T18:35:55.288165+0000","last_clean":"2023-06-23T18:35:55.288165+0000","last_became_active":"2023-06-23T18:35:53.534450+0000","last_became_peered":"2023-06-23T18:35:53.534450+0000","last_unstale":"2023-06-23T18:35:55.288165+0000","last_undegraded":"2023-06-23T18:35:55.288165+0000","last_fullsized":"2023-06-23T18:35:55.288165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527020+0000","last_change":"2023-06-23T18:35:53.533580+0000","last_active":"2023-06-23T18:35:54.527020+0000","last_peered":"2023-06-23T18:35:54.527020+0000","last_clean":"2023-06-23T18:35:54.527020+0000","last_became_active":"2023-06-23T18:35:53.533273+0000","last_became_peered":"2023-06-23T18:35:53.533273+0000","last_unstale":"2023-06-23T18:35:54.527020+0000","last_undegraded":"2023-06-23T18:35:54.527020+0000","last_fullsized":"2023-06-23T18:35:54.527020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526443+0000","last_change":"2023-06-23T18:35:53.537638+0000","last_active":"2023-06-23T18:35:54.526443+0000","last_peered":"2023-06-23T18:35:54.526443+0000","last_clean":"2023-06-23T18:35:54.526443+0000","last_became_active":"2023-06-23T18:35:53.537351+0000","last_became_peered":"2023-06-23T18:35:53.537351+0000","last_unstale":"2023-06-23T18:35:54.526443+0000","last_undegraded":"2023-06-23T18:35:54.526443+0000","last_fullsized":"2023-06-23T18:35:54.526443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:25:31.659636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288045+0000","last_change":"2023-06-23T18:35:53.530072+0000","last_active":"2023-06-23T18:35:55.288045+0000","last_peered":"2023-06-23T18:35:55.288045+0000","last_clean":"2023-06-23T18:35:55.288045+0000","last_became_active":"2023-06-23T18:35:53.529676+0000","last_became_peered":"2023-06-23T18:35:53.529676+0000","last_unstale":"2023-06-23T18:35:55.288045+0000","last_undegraded":"2023-06-23T18:35:55.288045+0000","last_fullsized":"2023-06-23T18:35:55.288045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:04:21.490439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287613+0000","last_change":"2023-06-23T18:35:53.535447+0000","last_active":"2023-06-23T18:35:55.287613+0000","last_peered":"2023-06-23T18:35:55.287613+0000","last_clean":"2023-06-23T18:35:55.287613+0000","last_became_active":"2023-06-23T18:35:53.535211+0000","last_became_peered":"2023-06-23T18:35:53.535211+0000","last_unstale":"2023-06-23T18:35:55.287613+0000","last_undegraded":"2023-06-23T18:35:55.287613+0000","last_fullsized":"2023-06-23T18:35:55.287613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:29:59.349736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526832+0000","last_change":"2023-06-23T18:35:53.530753+0000","last_active":"2023-06-23T18:35:54.526832+0000","last_peered":"2023-06-23T18:35:54.526832+0000","last_clean":"2023-06-23T18:35:54.526832+0000","last_became_active":"2023-06-23T18:35:53.530428+0000","last_became_peered":"2023-06-23T18:35:53.530428+0000","last_unstale":"2023-06-23T18:35:54.526832+0000","last_undegraded":"2023-06-23T18:35:54.526832+0000","last_fullsized":"2023-06-23T18:35:54.526832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:00.677969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794039+0000","last_change":"2023-06-23T18:35:53.540712+0000","last_active":"2023-06-23T18:35:54.794039+0000","last_peered":"2023-06-23T18:35:54.794039+0000","last_clean":"2023-06-23T18:35:54.794039+0000","last_became_active":"2023-06-23T18:35:53.540284+0000","last_became_peered":"2023-06-23T18:35:53.540284+0000","last_unstale":"2023-06-23T18:35:54.794039+0000","last_undegraded":"2023-06-23T18:35:54.794039+0000","last_fullsized":"2023-06-23T18:35:54.794039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:12.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525689+0000","last_change":"2023-06-23T18:35:53.524624+0000","last_active":"2023-06-23T18:35:54.525689+0000","last_peered":"2023-06-23T18:35:54.525689+0000","last_clean":"2023-06-23T18:35:54.525689+0000","last_became_active":"2023-06-23T18:35:53.524336+0000","last_became_peered":"2023-06-23T18:35:53.524336+0000","last_unstale":"2023-06-23T18:35:54.525689+0000","last_undegraded":"2023-06-23T18:35:54.525689+0000","last_fullsized":"2023-06-23T18:35:54.525689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:16.869108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288289+0000","last_change":"2023-06-23T18:35:53.534752+0000","last_active":"2023-06-23T18:35:55.288289+0000","last_peered":"2023-06-23T18:35:55.288289+0000","last_clean":"2023-06-23T18:35:55.288289+0000","last_became_active":"2023-06-23T18:35:53.534491+0000","last_became_peered":"2023-06-23T18:35:53.534491+0000","last_unstale":"2023-06-23T18:35:55.288289+0000","last_undegraded":"2023-06-23T18:35:55.288289+0000","last_fullsized":"2023-06-23T18:35:55.288289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:16.869108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526631+0000","last_change":"2023-06-23T18:35:53.530226+0000","last_active":"2023-06-23T18:35:54.526631+0000","last_peered":"2023-06-23T18:35:54.526631+0000","last_clean":"2023-06-23T18:35:54.526631+0000","last_became_active":"2023-06-23T18:35:53.529956+0000","last_became_peered":"2023-06-23T18:35:53.529956+0000","last_unstale":"2023-06-23T18:35:54.526631+0000","last_undegraded":"2023-06-23T18:35:54.526631+0000","last_fullsized":"2023-06-23T18:35:54.526631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:12.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287300+0000","last_change":"2023-06-23T18:35:53.540646+0000","last_active":"2023-06-23T18:35:55.287300+0000","last_peered":"2023-06-23T18:35:55.287300+0000","last_clean":"2023-06-23T18:35:55.287300+0000","last_became_active":"2023-06-23T18:35:53.540370+0000","last_became_peered":"2023-06-23T18:35:53.540370+0000","last_unstale":"2023-06-23T18:35:55.287300+0000","last_undegraded":"2023-06-23T18:35:55.287300+0000","last_fullsized":"2023-06-23T18:35:55.287300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:38:52.875168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526208+0000","last_change":"2023-06-23T18:35:53.539590+0000","last_active":"2023-06-23T18:35:54.526208+0000","last_peered":"2023-06-23T18:35:54.526208+0000","last_clean":"2023-06-23T18:35:54.526208+0000","last_became_active":"2023-06-23T18:35:53.539298+0000","last_became_peered":"2023-06-23T18:35:53.539298+0000","last_unstale":"2023-06-23T18:35:54.526208+0000","last_undegraded":"2023-06-23T18:35:54.526208+0000","last_fullsized":"2023-06-23T18:35:54.526208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:54:07.361405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794125+0000","last_change":"2023-06-23T18:35:53.543302+0000","last_active":"2023-06-23T18:35:54.794125+0000","last_peered":"2023-06-23T18:35:54.794125+0000","last_clean":"2023-06-23T18:35:54.794125+0000","last_became_active":"2023-06-23T18:35:53.542901+0000","last_became_peered":"2023-06-23T18:35:53.542901+0000","last_unstale":"2023-06-23T18:35:54.794125+0000","last_undegraded":"2023-06-23T18:35:54.794125+0000","last_fullsized":"2023-06-23T18:35:54.794125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:26:44.849536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287754+0000","last_change":"2023-06-23T18:35:53.525002+0000","last_active":"2023-06-23T18:35:55.287754+0000","last_peered":"2023-06-23T18:35:55.287754+0000","last_clean":"2023-06-23T18:35:55.287754+0000","last_became_active":"2023-06-23T18:35:53.524694+0000","last_became_peered":"2023-06-23T18:35:53.524694+0000","last_unstale":"2023-06-23T18:35:55.287754+0000","last_undegraded":"2023-06-23T18:35:55.287754+0000","last_fullsized":"2023-06-23T18:35:55.287754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:28.977806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288309+0000","last_change":"2023-06-23T18:35:53.532165+0000","last_active":"2023-06-23T18:35:55.288309+0000","last_peered":"2023-06-23T18:35:55.288309+0000","last_clean":"2023-06-23T18:35:55.288309+0000","last_became_active":"2023-06-23T18:35:53.531883+0000","last_became_peered":"2023-06-23T18:35:53.531883+0000","last_unstale":"2023-06-23T18:35:55.288309+0000","last_undegraded":"2023-06-23T18:35:55.288309+0000","last_fullsized":"2023-06-23T18:35:55.288309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793736+0000","last_change":"2023-06-23T18:35:53.543926+0000","last_active":"2023-06-23T18:35:54.793736+0000","last_peered":"2023-06-23T18:35:54.793736+0000","last_clean":"2023-06-23T18:35:54.793736+0000","last_became_active":"2023-06-23T18:35:53.543692+0000","last_became_peered":"2023-06-23T18:35:53.543692+0000","last_unstale":"2023-06-23T18:35:54.793736+0000","last_undegraded":"2023-06-23T18:35:54.793736+0000","last_fullsized":"2023-06-23T18:35:54.793736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:19:44.074507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527502+0000","last_change":"2023-06-23T18:35:53.539274+0000","last_active":"2023-06-23T18:35:54.527502+0000","last_peered":"2023-06-23T18:35:54.527502+0000","last_clean":"2023-06-23T18:35:54.527502+0000","last_became_active":"2023-06-23T18:35:53.539000+0000","last_became_peered":"2023-06-23T18:35:53.539000+0000","last_unstale":"2023-06-23T18:35:54.527502+0000","last_undegraded":"2023-06-23T18:35:54.527502+0000","last_fullsized":"2023-06-23T18:35:54.527502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:00.677969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527478+0000","last_change":"2023-06-23T18:35:53.532343+0000","last_active":"2023-06-23T18:35:54.527478+0000","last_peered":"2023-06-23T18:35:54.527478+0000","last_clean":"2023-06-23T18:35:54.527478+0000","last_became_active":"2023-06-23T18:35:53.532137+0000","last_became_peered":"2023-06-23T18:35:53.532137+0000","last_unstale":"2023-06-23T18:35:54.527478+0000","last_undegraded":"2023-06-23T18:35:54.527478+0000","last_fullsized":"2023-06-23T18:35:54.527478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:41.966301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526775+0000","last_change":"2023-06-23T18:35:53.541546+0000","last_active":"2023-06-23T18:35:54.526775+0000","last_peered":"2023-06-23T18:35:54.526775+0000","last_clean":"2023-06-23T18:35:54.526775+0000","last_became_active":"2023-06-23T18:35:53.541285+0000","last_became_peered":"2023-06-23T18:35:53.541285+0000","last_unstale":"2023-06-23T18:35:54.526775+0000","last_undegraded":"2023-06-23T18:35:54.526775+0000","last_fullsized":"2023-06-23T18:35:54.526775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:41.966301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527025+0000","last_change":"2023-06-23T18:35:53.130775+0000","last_active":"2023-06-23T18:35:54.527025+0000","last_peered":"2023-06-23T18:35:54.527025+0000","last_clean":"2023-06-23T18:35:54.527025+0000","last_became_active":"2023-06-23T18:35:53.130473+0000","last_became_peered":"2023-06-23T18:35:53.130473+0000","last_unstale":"2023-06-23T18:35:54.527025+0000","last_undegraded":"2023-06-23T18:35:54.527025+0000","last_fullsized":"2023-06-23T18:35:54.527025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:26:43.847769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288817+0000","last_change":"2023-06-23T18:35:53.530941+0000","last_active":"2023-06-23T18:35:55.288817+0000","last_peered":"2023-06-23T18:35:55.288817+0000","last_clean":"2023-06-23T18:35:55.288817+0000","last_became_active":"2023-06-23T18:35:53.530683+0000","last_became_peered":"2023-06-23T18:35:53.530683+0000","last_unstale":"2023-06-23T18:35:55.288817+0000","last_undegraded":"2023-06-23T18:35:55.288817+0000","last_fullsized":"2023-06-23T18:35:55.288817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:19:44.074507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527326+0000","last_change":"2023-06-23T18:35:53.132155+0000","last_active":"2023-06-23T18:35:54.527326+0000","last_peered":"2023-06-23T18:35:54.527326+0000","last_clean":"2023-06-23T18:35:54.527326+0000","last_became_active":"2023-06-23T18:35:53.131785+0000","last_became_peered":"2023-06-23T18:35:53.131785+0000","last_unstale":"2023-06-23T18:35:54.527326+0000","last_undegraded":"2023-06-23T18:35:54.527326+0000","last_fullsized":"2023-06-23T18:35:54.527326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:51.656412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288482+0000","last_change":"2023-06-23T18:35:53.537294+0000","last_active":"2023-06-23T18:35:55.288482+0000","last_peered":"2023-06-23T18:35:55.288482+0000","last_clean":"2023-06-23T18:35:55.288482+0000","last_became_active":"2023-06-23T18:35:53.536977+0000","last_became_peered":"2023-06-23T18:35:53.536977+0000","last_unstale":"2023-06-23T18:35:55.288482+0000","last_undegraded":"2023-06-23T18:35:55.288482+0000","last_fullsized":"2023-06-23T18:35:55.288482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:30.583311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288741+0000","last_change":"2023-06-23T18:35:53.135828+0000","last_active":"2023-06-23T18:35:55.288741+0000","last_peered":"2023-06-23T18:35:55.288741+0000","last_clean":"2023-06-23T18:35:55.288741+0000","last_became_active":"2023-06-23T18:35:53.135453+0000","last_became_peered":"2023-06-23T18:35:53.135453+0000","last_unstale":"2023-06-23T18:35:55.288741+0000","last_undegraded":"2023-06-23T18:35:55.288741+0000","last_fullsized":"2023-06-23T18:35:55.288741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:54:06.359638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257486+0000","last_change":"2023-06-23T18:35:53.535934+0000","last_active":"2023-06-23T18:35:55.257486+0000","last_peered":"2023-06-23T18:35:55.257486+0000","last_clean":"2023-06-23T18:35:55.257486+0000","last_became_active":"2023-06-23T18:35:53.535505+0000","last_became_peered":"2023-06-23T18:35:53.535505+0000","last_unstale":"2023-06-23T18:35:55.257486+0000","last_undegraded":"2023-06-23T18:35:55.257486+0000","last_fullsized":"2023-06-23T18:35:55.257486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288360+0000","last_change":"2023-06-23T18:35:53.130045+0000","last_active":"2023-06-23T18:35:55.288360+0000","last_peered":"2023-06-23T18:35:55.288360+0000","last_clean":"2023-06-23T18:35:55.288360+0000","last_became_active":"2023-06-23T18:35:53.129374+0000","last_became_peered":"2023-06-23T18:35:53.129374+0000","last_unstale":"2023-06-23T18:35:55.288360+0000","last_undegraded":"2023-06-23T18:35:55.288360+0000","last_fullsized":"2023-06-23T18:35:55.288360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:30.042205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288533+0000","last_change":"2023-06-23T18:35:53.540785+0000","last_active":"2023-06-23T18:35:55.288533+0000","last_peered":"2023-06-23T18:35:55.288533+0000","last_clean":"2023-06-23T18:35:55.288533+0000","last_became_active":"2023-06-23T18:35:53.540511+0000","last_became_peered":"2023-06-23T18:35:53.540511+0000","last_unstale":"2023-06-23T18:35:55.288533+0000","last_undegraded":"2023-06-23T18:35:55.288533+0000","last_fullsized":"2023-06-23T18:35:55.288533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:17.520121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526768+0000","last_change":"2023-06-23T18:35:53.133079+0000","last_active":"2023-06-23T18:35:54.526768+0000","last_peered":"2023-06-23T18:35:54.526768+0000","last_clean":"2023-06-23T18:35:54.526768+0000","last_became_active":"2023-06-23T18:35:53.132662+0000","last_became_peered":"2023-06-23T18:35:53.132662+0000","last_unstale":"2023-06-23T18:35:54.526768+0000","last_undegraded":"2023-06-23T18:35:54.526768+0000","last_fullsized":"2023-06-23T18:35:54.526768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:06.645556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288606+0000","last_change":"2023-06-23T18:35:53.541521+0000","last_active":"2023-06-23T18:35:55.288606+0000","last_peered":"2023-06-23T18:35:55.288606+0000","last_clean":"2023-06-23T18:35:55.288606+0000","last_became_active":"2023-06-23T18:35:53.541265+0000","last_became_peered":"2023-06-23T18:35:53.541265+0000","last_unstale":"2023-06-23T18:35:55.288606+0000","last_undegraded":"2023-06-23T18:35:55.288606+0000","last_fullsized":"2023-06-23T18:35:55.288606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:39.018447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793951+0000","last_change":"2023-06-23T18:35:53.131904+0000","last_active":"2023-06-23T18:35:54.793951+0000","last_peered":"2023-06-23T18:35:54.793951+0000","last_clean":"2023-06-23T18:35:54.793951+0000","last_became_active":"2023-06-23T18:35:53.131581+0000","last_became_peered":"2023-06-23T18:35:53.131581+0000","last_unstale":"2023-06-23T18:35:54.793951+0000","last_undegraded":"2023-06-23T18:35:54.793951+0000","last_fullsized":"2023-06-23T18:35:54.793951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:25:30.657869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527297+0000","last_change":"2023-06-23T18:35:53.542076+0000","last_active":"2023-06-23T18:35:54.527297+0000","last_peered":"2023-06-23T18:35:54.527297+0000","last_clean":"2023-06-23T18:35:54.527297+0000","last_became_active":"2023-06-23T18:35:53.541814+0000","last_became_peered":"2023-06-23T18:35:53.541814+0000","last_unstale":"2023-06-23T18:35:54.527297+0000","last_undegraded":"2023-06-23T18:35:54.527297+0000","last_fullsized":"2023-06-23T18:35:54.527297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:40:01.768196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526932+0000","last_change":"2023-06-23T18:35:53.131165+0000","last_active":"2023-06-23T18:35:54.526932+0000","last_peered":"2023-06-23T18:35:54.526932+0000","last_clean":"2023-06-23T18:35:54.526932+0000","last_became_active":"2023-06-23T18:35:53.130890+0000","last_became_peered":"2023-06-23T18:35:53.130890+0000","last_unstale":"2023-06-23T18:35:54.526932+0000","last_undegraded":"2023-06-23T18:35:54.526932+0000","last_fullsized":"2023-06-23T18:35:54.526932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:16:18.754209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526252+0000","last_change":"2023-06-23T18:35:53.541719+0000","last_active":"2023-06-23T18:35:54.526252+0000","last_peered":"2023-06-23T18:35:54.526252+0000","last_clean":"2023-06-23T18:35:54.526252+0000","last_became_active":"2023-06-23T18:35:53.541456+0000","last_became_peered":"2023-06-23T18:35:53.541456+0000","last_unstale":"2023-06-23T18:35:54.526252+0000","last_undegraded":"2023-06-23T18:35:54.526252+0000","last_fullsized":"2023-06-23T18:35:54.526252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:30:45.259696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794066+0000","last_change":"2023-06-23T18:35:53.136124+0000","last_active":"2023-06-23T18:35:54.794066+0000","last_peered":"2023-06-23T18:35:54.794066+0000","last_clean":"2023-06-23T18:35:54.794066+0000","last_became_active":"2023-06-23T18:35:53.135736+0000","last_became_peered":"2023-06-23T18:35:53.135736+0000","last_unstale":"2023-06-23T18:35:54.794066+0000","last_undegraded":"2023-06-23T18:35:54.794066+0000","last_fullsized":"2023-06-23T18:35:54.794066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:45:16.668072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.289031+0000","last_change":"2023-06-23T18:35:53.540586+0000","last_active":"2023-06-23T18:35:55.289031+0000","last_peered":"2023-06-23T18:35:55.289031+0000","last_clean":"2023-06-23T18:35:55.289031+0000","last_became_active":"2023-06-23T18:35:53.540355+0000","last_became_peered":"2023-06-23T18:35:53.540355+0000","last_unstale":"2023-06-23T18:35:55.289031+0000","last_undegraded":"2023-06-23T18:35:55.289031+0000","last_fullsized":"2023-06-23T18:35:55.289031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:02:35.233797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287933+0000","last_change":"2023-06-23T18:35:53.133113+0000","last_active":"2023-06-23T18:35:55.287933+0000","last_peered":"2023-06-23T18:35:55.287933+0000","last_clean":"2023-06-23T18:35:55.287933+0000","last_became_active":"2023-06-23T18:35:53.132833+0000","last_became_peered":"2023-06-23T18:35:53.132833+0000","last_unstale":"2023-06-23T18:35:55.287933+0000","last_undegraded":"2023-06-23T18:35:55.287933+0000","last_fullsized":"2023-06-23T18:35:55.287933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:46:59.676202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527016+0000","last_change":"2023-06-23T18:35:53.538309+0000","last_active":"2023-06-23T18:35:54.527016+0000","last_peered":"2023-06-23T18:35:54.527016+0000","last_clean":"2023-06-23T18:35:54.527016+0000","last_became_active":"2023-06-23T18:35:53.538037+0000","last_became_peered":"2023-06-23T18:35:53.538037+0000","last_unstale":"2023-06-23T18:35:54.527016+0000","last_undegraded":"2023-06-23T18:35:54.527016+0000","last_fullsized":"2023-06-23T18:35:54.527016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793844+0000","last_change":"2023-06-23T18:35:53.131958+0000","last_active":"2023-06-23T18:35:54.793844+0000","last_peered":"2023-06-23T18:35:54.793844+0000","last_clean":"2023-06-23T18:35:54.793844+0000","last_became_active":"2023-06-23T18:35:53.131643+0000","last_became_peered":"2023-06-23T18:35:53.131643+0000","last_unstale":"2023-06-23T18:35:54.793844+0000","last_undegraded":"2023-06-23T18:35:54.793844+0000","last_fullsized":"2023-06-23T18:35:54.793844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:57.730157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288109+0000","last_change":"2023-06-23T18:35:53.533318+0000","last_active":"2023-06-23T18:35:55.288109+0000","last_peered":"2023-06-23T18:35:55.288109+0000","last_clean":"2023-06-23T18:35:55.288109+0000","last_became_active":"2023-06-23T18:35:53.533081+0000","last_became_peered":"2023-06-23T18:35:53.533081+0000","last_unstale":"2023-06-23T18:35:55.288109+0000","last_undegraded":"2023-06-23T18:35:55.288109+0000","last_fullsized":"2023-06-23T18:35:55.288109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257090+0000","last_change":"2023-06-23T18:35:53.135982+0000","last_active":"2023-06-23T18:35:55.257090+0000","last_peered":"2023-06-23T18:35:55.257090+0000","last_clean":"2023-06-23T18:35:55.257090+0000","last_became_active":"2023-06-23T18:35:53.135591+0000","last_became_peered":"2023-06-23T18:35:53.135591+0000","last_unstale":"2023-06-23T18:35:55.257090+0000","last_undegraded":"2023-06-23T18:35:55.257090+0000","last_fullsized":"2023-06-23T18:35:55.257090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:40:00.766429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288988+0000","last_change":"2023-06-23T18:35:53.541883+0000","last_active":"2023-06-23T18:35:55.288988+0000","last_peered":"2023-06-23T18:35:55.288988+0000","last_clean":"2023-06-23T18:35:55.288988+0000","last_became_active":"2023-06-23T18:35:53.541630+0000","last_became_peered":"2023-06-23T18:35:53.541630+0000","last_unstale":"2023-06-23T18:35:55.288988+0000","last_undegraded":"2023-06-23T18:35:55.288988+0000","last_fullsized":"2023-06-23T18:35:55.288988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:08:04.256786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793955+0000","last_change":"2023-06-23T18:35:53.132583+0000","last_active":"2023-06-23T18:35:54.793955+0000","last_peered":"2023-06-23T18:35:54.793955+0000","last_clean":"2023-06-23T18:35:54.793955+0000","last_became_active":"2023-06-23T18:35:53.132142+0000","last_became_peered":"2023-06-23T18:35:53.132142+0000","last_unstale":"2023-06-23T18:35:54.793955+0000","last_undegraded":"2023-06-23T18:35:54.793955+0000","last_fullsized":"2023-06-23T18:35:54.793955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:02:34.232029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526009+0000","last_change":"2023-06-23T18:35:53.537549+0000","last_active":"2023-06-23T18:35:54.526009+0000","last_peered":"2023-06-23T18:35:54.526009+0000","last_clean":"2023-06-23T18:35:54.526009+0000","last_became_active":"2023-06-23T18:35:53.537281+0000","last_became_peered":"2023-06-23T18:35:53.537281+0000","last_unstale":"2023-06-23T18:35:54.526009+0000","last_undegraded":"2023-06-23T18:35:54.526009+0000","last_fullsized":"2023-06-23T18:35:54.526009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:17.016870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794202+0000","last_change":"2023-06-23T18:35:53.131954+0000","last_active":"2023-06-23T18:35:54.794202+0000","last_peered":"2023-06-23T18:35:54.794202+0000","last_clean":"2023-06-23T18:35:54.794202+0000","last_became_active":"2023-06-23T18:35:53.131624+0000","last_became_peered":"2023-06-23T18:35:53.131624+0000","last_unstale":"2023-06-23T18:35:54.794202+0000","last_undegraded":"2023-06-23T18:35:54.794202+0000","last_fullsized":"2023-06-23T18:35:54.794202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:16:18.754209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525847+0000","last_change":"2023-06-23T18:35:53.535467+0000","last_active":"2023-06-23T18:35:54.525847+0000","last_peered":"2023-06-23T18:35:54.525847+0000","last_clean":"2023-06-23T18:35:54.525847+0000","last_became_active":"2023-06-23T18:35:53.535187+0000","last_became_peered":"2023-06-23T18:35:53.535187+0000","last_unstale":"2023-06-23T18:35:54.525847+0000","last_undegraded":"2023-06-23T18:35:54.525847+0000","last_fullsized":"2023-06-23T18:35:54.525847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:31.043972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793955+0000","last_change":"2023-06-23T18:35:53.132445+0000","last_active":"2023-06-23T18:35:54.793955+0000","last_peered":"2023-06-23T18:35:54.793955+0000","last_clean":"2023-06-23T18:35:54.793955+0000","last_became_active":"2023-06-23T18:35:53.132108+0000","last_became_peered":"2023-06-23T18:35:53.132108+0000","last_unstale":"2023-06-23T18:35:54.793955+0000","last_undegraded":"2023-06-23T18:35:54.793955+0000","last_fullsized":"2023-06-23T18:35:54.793955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:44:02.521500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257388+0000","last_change":"2023-06-23T18:35:53.539086+0000","last_active":"2023-06-23T18:35:55.257388+0000","last_peered":"2023-06-23T18:35:55.257388+0000","last_clean":"2023-06-23T18:35:55.257388+0000","last_became_active":"2023-06-23T18:35:53.538755+0000","last_became_peered":"2023-06-23T18:35:53.538755+0000","last_unstale":"2023-06-23T18:35:55.257388+0000","last_undegraded":"2023-06-23T18:35:55.257388+0000","last_fullsized":"2023-06-23T18:35:55.257388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:07.647324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527056+0000","last_change":"2023-06-23T18:35:53.128100+0000","last_active":"2023-06-23T18:35:54.527056+0000","last_peered":"2023-06-23T18:35:54.527056+0000","last_clean":"2023-06-23T18:35:54.527056+0000","last_became_active":"2023-06-23T18:35:53.127782+0000","last_became_peered":"2023-06-23T18:35:53.127782+0000","last_unstale":"2023-06-23T18:35:54.527056+0000","last_undegraded":"2023-06-23T18:35:54.527056+0000","last_fullsized":"2023-06-23T18:35:54.527056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:15.867340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287463+0000","last_change":"2023-06-23T18:35:53.524781+0000","last_active":"2023-06-23T18:35:55.287463+0000","last_peered":"2023-06-23T18:35:55.287463+0000","last_clean":"2023-06-23T18:35:55.287463+0000","last_became_active":"2023-06-23T18:35:53.524526+0000","last_became_peered":"2023-06-23T18:35:53.524526+0000","last_unstale":"2023-06-23T18:35:55.287463+0000","last_undegraded":"2023-06-23T18:35:55.287463+0000","last_fullsized":"2023-06-23T18:35:55.287463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:40:01.768196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288801+0000","last_change":"2023-06-23T18:35:53.133550+0000","last_active":"2023-06-23T18:35:55.288801+0000","last_peered":"2023-06-23T18:35:55.288801+0000","last_clean":"2023-06-23T18:35:55.288801+0000","last_became_active":"2023-06-23T18:35:53.133287+0000","last_became_peered":"2023-06-23T18:35:53.133287+0000","last_unstale":"2023-06-23T18:35:55.288801+0000","last_undegraded":"2023-06-23T18:35:55.288801+0000","last_fullsized":"2023-06-23T18:35:55.288801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:30:44.257929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526733+0000","last_change":"2023-06-23T18:35:53.534074+0000","last_active":"2023-06-23T18:35:54.526733+0000","last_peered":"2023-06-23T18:35:54.526733+0000","last_clean":"2023-06-23T18:35:54.526733+0000","last_became_active":"2023-06-23T18:35:53.533789+0000","last_became_peered":"2023-06-23T18:35:53.533789+0000","last_unstale":"2023-06-23T18:35:54.526733+0000","last_undegraded":"2023-06-23T18:35:54.526733+0000","last_fullsized":"2023-06-23T18:35:54.526733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:58.731924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257244+0000","last_change":"2023-06-23T18:35:53.126156+0000","last_active":"2023-06-23T18:35:55.257244+0000","last_peered":"2023-06-23T18:35:55.257244+0000","last_clean":"2023-06-23T18:35:55.257244+0000","last_became_active":"2023-06-23T18:35:53.125640+0000","last_became_peered":"2023-06-23T18:35:53.125640+0000","last_unstale":"2023-06-23T18:35:55.257244+0000","last_undegraded":"2023-06-23T18:35:55.257244+0000","last_fullsized":"2023-06-23T18:35:55.257244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:16.015103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526631+0000","last_change":"2023-06-23T18:35:53.529926+0000","last_active":"2023-06-23T18:35:54.526631+0000","last_peered":"2023-06-23T18:35:54.526631+0000","last_clean":"2023-06-23T18:35:54.526631+0000","last_became_active":"2023-06-23T18:35:53.529619+0000","last_became_peered":"2023-06-23T18:35:53.529619+0000","last_unstale":"2023-06-23T18:35:54.526631+0000","last_undegraded":"2023-06-23T18:35:54.526631+0000","last_fullsized":"2023-06-23T18:35:54.526631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:44:03.523267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526406+0000","last_change":"2023-06-23T18:35:53.134550+0000","last_active":"2023-06-23T18:35:54.526406+0000","last_peered":"2023-06-23T18:35:54.526406+0000","last_clean":"2023-06-23T18:35:54.526406+0000","last_became_active":"2023-06-23T18:35:53.132753+0000","last_became_peered":"2023-06-23T18:35:53.132753+0000","last_unstale":"2023-06-23T18:35:54.526406+0000","last_undegraded":"2023-06-23T18:35:54.526406+0000","last_fullsized":"2023-06-23T18:35:54.526406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:19:43.072740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526469+0000","last_change":"2023-06-23T18:35:53.541907+0000","last_active":"2023-06-23T18:35:54.526469+0000","last_peered":"2023-06-23T18:35:54.526469+0000","last_clean":"2023-06-23T18:35:54.526469+0000","last_became_active":"2023-06-23T18:35:53.541545+0000","last_became_peered":"2023-06-23T18:35:53.541545+0000","last_unstale":"2023-06-23T18:35:54.526469+0000","last_undegraded":"2023-06-23T18:35:54.526469+0000","last_fullsized":"2023-06-23T18:35:54.526469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:02:35.233797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288368+0000","last_change":"2023-06-23T18:35:53.130943+0000","last_active":"2023-06-23T18:35:55.288368+0000","last_peered":"2023-06-23T18:35:55.288368+0000","last_clean":"2023-06-23T18:35:55.288368+0000","last_became_active":"2023-06-23T18:35:53.130544+0000","last_became_peered":"2023-06-23T18:35:53.130544+0000","last_unstale":"2023-06-23T18:35:55.288368+0000","last_undegraded":"2023-06-23T18:35:55.288368+0000","last_fullsized":"2023-06-23T18:35:55.288368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:18.373567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527635+0000","last_change":"2023-06-23T18:35:53.535355+0000","last_active":"2023-06-23T18:35:54.527635+0000","last_peered":"2023-06-23T18:35:54.527635+0000","last_clean":"2023-06-23T18:35:54.527635+0000","last_became_active":"2023-06-23T18:35:53.535087+0000","last_became_peered":"2023-06-23T18:35:53.535087+0000","last_unstale":"2023-06-23T18:35:54.527635+0000","last_undegraded":"2023-06-23T18:35:54.527635+0000","last_fullsized":"2023-06-23T18:35:54.527635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288978+0000","last_change":"2023-06-23T18:35:53.133469+0000","last_active":"2023-06-23T18:35:55.288978+0000","last_peered":"2023-06-23T18:35:55.288978+0000","last_clean":"2023-06-23T18:35:55.288978+0000","last_became_active":"2023-06-23T18:35:53.133211+0000","last_became_peered":"2023-06-23T18:35:53.133211+0000","last_unstale":"2023-06-23T18:35:55.288978+0000","last_undegraded":"2023-06-23T18:35:55.288978+0000","last_fullsized":"2023-06-23T18:35:55.288978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:57:45.216699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526357+0000","last_change":"2023-06-23T18:35:48.920009+0000","last_active":"2023-06-23T18:35:54.526357+0000","last_peered":"2023-06-23T18:35:54.526357+0000","last_clean":"2023-06-23T18:35:54.526357+0000","last_became_active":"2023-06-23T18:35:48.919686+0000","last_became_peered":"2023-06-23T18:35:48.919686+0000","last_unstale":"2023-06-23T18:35:54.526357+0000","last_undegraded":"2023-06-23T18:35:54.526357+0000","last_fullsized":"2023-06-23T18:35:54.526357+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:57:40.062937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288250+0000","last_change":"2023-06-23T18:35:53.538294+0000","last_active":"2023-06-23T18:35:55.288250+0000","last_peered":"2023-06-23T18:35:55.288250+0000","last_clean":"2023-06-23T18:35:55.288250+0000","last_became_active":"2023-06-23T18:35:53.537996+0000","last_became_peered":"2023-06-23T18:35:53.537996+0000","last_unstale":"2023-06-23T18:35:55.288250+0000","last_undegraded":"2023-06-23T18:35:55.288250+0000","last_fullsized":"2023-06-23T18:35:55.288250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288977+0000","last_change":"2023-06-23T18:35:47.962164+0000","last_active":"2023-06-23T18:35:55.288977+0000","last_peered":"2023-06-23T18:35:55.288977+0000","last_clean":"2023-06-23T18:35:55.288977+0000","last_became_active":"2023-06-23T18:35:47.961790+0000","last_became_peered":"2023-06-23T18:35:47.961790+0000","last_unstale":"2023-06-23T18:35:55.288977+0000","last_undegraded":"2023-06-23T18:35:55.288977+0000","last_fullsized":"2023-06-23T18:35:55.288977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:25:25.504108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288539+0000","last_change":"2023-06-23T18:35:53.126850+0000","last_active":"2023-06-23T18:35:55.288539+0000","last_peered":"2023-06-23T18:35:55.288539+0000","last_clean":"2023-06-23T18:35:55.288539+0000","last_became_active":"2023-06-23T18:35:53.126560+0000","last_became_peered":"2023-06-23T18:35:53.126560+0000","last_unstale":"2023-06-23T18:35:55.288539+0000","last_undegraded":"2023-06-23T18:35:55.288539+0000","last_fullsized":"2023-06-23T18:35:55.288539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:38.016680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526607+0000","last_change":"2023-06-23T18:35:53.531624+0000","last_active":"2023-06-23T18:35:54.526607+0000","last_peered":"2023-06-23T18:35:54.526607+0000","last_clean":"2023-06-23T18:35:54.526607+0000","last_became_active":"2023-06-23T18:35:53.531197+0000","last_became_peered":"2023-06-23T18:35:53.531197+0000","last_unstale":"2023-06-23T18:35:54.526607+0000","last_undegraded":"2023-06-23T18:35:54.526607+0000","last_fullsized":"2023-06-23T18:35:54.526607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:17.520121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527435+0000","last_change":"2023-06-23T18:35:47.965867+0000","last_active":"2023-06-23T18:35:54.527435+0000","last_peered":"2023-06-23T18:35:54.527435+0000","last_clean":"2023-06-23T18:35:54.527435+0000","last_became_active":"2023-06-23T18:35:47.965580+0000","last_became_peered":"2023-06-23T18:35:47.965580+0000","last_unstale":"2023-06-23T18:35:54.527435+0000","last_undegraded":"2023-06-23T18:35:54.527435+0000","last_fullsized":"2023-06-23T18:35:54.527435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:01.491795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793869+0000","last_change":"2023-06-23T18:35:47.968534+0000","last_active":"2023-06-23T18:35:54.793869+0000","last_peered":"2023-06-23T18:35:54.793869+0000","last_clean":"2023-06-23T18:35:54.793869+0000","last_became_active":"2023-06-23T18:35:47.967508+0000","last_became_peered":"2023-06-23T18:35:47.967508+0000","last_unstale":"2023-06-23T18:35:54.793869+0000","last_undegraded":"2023-06-23T18:35:54.793869+0000","last_fullsized":"2023-06-23T18:35:54.793869+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:07:58.101257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287523+0000","last_change":"2023-06-23T18:35:53.126632+0000","last_active":"2023-06-23T18:35:55.287523+0000","last_peered":"2023-06-23T18:35:55.287523+0000","last_clean":"2023-06-23T18:35:55.287523+0000","last_became_active":"2023-06-23T18:35:53.126336+0000","last_became_peered":"2023-06-23T18:35:53.126336+0000","last_unstale":"2023-06-23T18:35:55.287523+0000","last_undegraded":"2023-06-23T18:35:55.287523+0000","last_fullsized":"2023-06-23T18:35:55.287523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:51.656412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288240+0000","last_change":"2023-06-23T18:35:53.524885+0000","last_active":"2023-06-23T18:35:55.288240+0000","last_peered":"2023-06-23T18:35:55.288240+0000","last_clean":"2023-06-23T18:35:55.288240+0000","last_became_active":"2023-06-23T18:35:53.524630+0000","last_became_peered":"2023-06-23T18:35:53.524630+0000","last_unstale":"2023-06-23T18:35:55.288240+0000","last_undegraded":"2023-06-23T18:35:55.288240+0000","last_fullsized":"2023-06-23T18:35:55.288240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793589+0000","last_change":"2023-06-23T18:35:48.736407+0000","last_active":"2023-06-23T18:35:54.793589+0000","last_peered":"2023-06-23T18:35:54.793589+0000","last_clean":"2023-06-23T18:35:54.793589+0000","last_became_active":"2023-06-23T18:35:48.735621+0000","last_became_peered":"2023-06-23T18:35:48.735621+0000","last_unstale":"2023-06-23T18:35:54.793589+0000","last_undegraded":"2023-06-23T18:35:54.793589+0000","last_fullsized":"2023-06-23T18:35:54.793589+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:30:39.104167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527092+0000","last_change":"2023-06-23T18:35:53.131817+0000","last_active":"2023-06-23T18:35:54.527092+0000","last_peered":"2023-06-23T18:35:54.527092+0000","last_clean":"2023-06-23T18:35:54.527092+0000","last_became_active":"2023-06-23T18:35:53.131549+0000","last_became_peered":"2023-06-23T18:35:53.131549+0000","last_unstale":"2023-06-23T18:35:54.527092+0000","last_undegraded":"2023-06-23T18:35:54.527092+0000","last_fullsized":"2023-06-23T18:35:54.527092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:04:00.010842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527627+0000","last_change":"2023-06-23T18:35:53.531485+0000","last_active":"2023-06-23T18:35:54.527627+0000","last_peered":"2023-06-23T18:35:54.527627+0000","last_clean":"2023-06-23T18:35:54.527627+0000","last_became_active":"2023-06-23T18:35:53.531276+0000","last_became_peered":"2023-06-23T18:35:53.531276+0000","last_unstale":"2023-06-23T18:35:54.527627+0000","last_undegraded":"2023-06-23T18:35:54.527627+0000","last_fullsized":"2023-06-23T18:35:54.527627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793552+0000","last_change":"2023-06-23T18:35:48.736449+0000","last_active":"2023-06-23T18:35:54.793552+0000","last_peered":"2023-06-23T18:35:54.793552+0000","last_clean":"2023-06-23T18:35:54.793552+0000","last_became_active":"2023-06-23T18:35:48.735664+0000","last_became_peered":"2023-06-23T18:35:48.735664+0000","last_unstale":"2023-06-23T18:35:54.793552+0000","last_undegraded":"2023-06-23T18:35:54.793552+0000","last_fullsized":"2023-06-23T18:35:54.793552+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:24.194443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288399+0000","last_change":"2023-06-23T18:35:53.127210+0000","last_active":"2023-06-23T18:35:55.288399+0000","last_peered":"2023-06-23T18:35:55.288399+0000","last_clean":"2023-06-23T18:35:55.288399+0000","last_became_active":"2023-06-23T18:35:53.126894+0000","last_became_peered":"2023-06-23T18:35:53.126894+0000","last_unstale":"2023-06-23T18:35:55.288399+0000","last_undegraded":"2023-06-23T18:35:55.288399+0000","last_fullsized":"2023-06-23T18:35:55.288399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:15.867340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257494+0000","last_change":"2023-06-23T18:35:53.539026+0000","last_active":"2023-06-23T18:35:55.257494+0000","last_peered":"2023-06-23T18:35:55.257494+0000","last_clean":"2023-06-23T18:35:55.257494+0000","last_became_active":"2023-06-23T18:35:53.538711+0000","last_became_peered":"2023-06-23T18:35:53.538711+0000","last_unstale":"2023-06-23T18:35:55.257494+0000","last_undegraded":"2023-06-23T18:35:55.257494+0000","last_fullsized":"2023-06-23T18:35:55.257494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:00.677969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526508+0000","last_change":"2023-06-23T18:35:48.735713+0000","last_active":"2023-06-23T18:35:54.526508+0000","last_peered":"2023-06-23T18:35:54.526508+0000","last_clean":"2023-06-23T18:35:54.526508+0000","last_became_active":"2023-06-23T18:35:48.735236+0000","last_became_peered":"2023-06-23T18:35:48.735236+0000","last_unstale":"2023-06-23T18:35:54.526508+0000","last_undegraded":"2023-06-23T18:35:54.526508+0000","last_fullsized":"2023-06-23T18:35:54.526508+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:22.822277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526010+0000","last_change":"2023-06-23T18:35:53.131500+0000","last_active":"2023-06-23T18:35:54.526010+0000","last_peered":"2023-06-23T18:35:54.526010+0000","last_clean":"2023-06-23T18:35:54.526010+0000","last_became_active":"2023-06-23T18:35:53.131222+0000","last_became_peered":"2023-06-23T18:35:53.131222+0000","last_unstale":"2023-06-23T18:35:54.526010+0000","last_undegraded":"2023-06-23T18:35:54.526010+0000","last_fullsized":"2023-06-23T18:35:54.526010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:38.016680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527523+0000","last_change":"2023-06-23T18:35:53.532882+0000","last_active":"2023-06-23T18:35:54.527523+0000","last_peered":"2023-06-23T18:35:54.527523+0000","last_clean":"2023-06-23T18:35:54.527523+0000","last_became_active":"2023-06-23T18:35:53.532615+0000","last_became_peered":"2023-06-23T18:35:53.532615+0000","last_unstale":"2023-06-23T18:35:54.527523+0000","last_undegraded":"2023-06-23T18:35:54.527523+0000","last_fullsized":"2023-06-23T18:35:54.527523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257469+0000","last_change":"2023-06-23T18:35:48.735631+0000","last_active":"2023-06-23T18:35:55.257469+0000","last_peered":"2023-06-23T18:35:55.257469+0000","last_clean":"2023-06-23T18:35:55.257469+0000","last_became_active":"2023-06-23T18:35:48.735255+0000","last_became_peered":"2023-06-23T18:35:48.735255+0000","last_unstale":"2023-06-23T18:35:55.257469+0000","last_undegraded":"2023-06-23T18:35:55.257469+0000","last_fullsized":"2023-06-23T18:35:55.257469+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:32:56.020041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527397+0000","last_change":"2023-06-23T18:35:53.132237+0000","last_active":"2023-06-23T18:35:54.527397+0000","last_peered":"2023-06-23T18:35:54.527397+0000","last_clean":"2023-06-23T18:35:54.527397+0000","last_became_active":"2023-06-23T18:35:53.131968+0000","last_became_peered":"2023-06-23T18:35:53.131968+0000","last_unstale":"2023-06-23T18:35:54.527397+0000","last_undegraded":"2023-06-23T18:35:54.527397+0000","last_fullsized":"2023-06-23T18:35:54.527397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:27.976038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287780+0000","last_change":"2023-06-23T18:35:53.536675+0000","last_active":"2023-06-23T18:35:55.287780+0000","last_peered":"2023-06-23T18:35:55.287780+0000","last_clean":"2023-06-23T18:35:55.287780+0000","last_became_active":"2023-06-23T18:35:53.535843+0000","last_became_peered":"2023-06-23T18:35:53.535843+0000","last_unstale":"2023-06-23T18:35:55.287780+0000","last_undegraded":"2023-06-23T18:35:55.287780+0000","last_fullsized":"2023-06-23T18:35:55.287780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:07.647324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257359+0000","last_change":"2023-06-23T18:35:47.970359+0000","last_active":"2023-06-23T18:35:55.257359+0000","last_peered":"2023-06-23T18:35:55.257359+0000","last_clean":"2023-06-23T18:35:55.257359+0000","last_became_active":"2023-06-23T18:35:47.970118+0000","last_became_peered":"2023-06-23T18:35:47.970118+0000","last_unstale":"2023-06-23T18:35:55.257359+0000","last_undegraded":"2023-06-23T18:35:55.257359+0000","last_fullsized":"2023-06-23T18:35:55.257359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:35.810772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527072+0000","last_change":"2023-06-23T18:35:53.132066+0000","last_active":"2023-06-23T18:35:54.527072+0000","last_peered":"2023-06-23T18:35:54.527072+0000","last_clean":"2023-06-23T18:35:54.527072+0000","last_became_active":"2023-06-23T18:35:53.131788+0000","last_became_peered":"2023-06-23T18:35:53.131788+0000","last_unstale":"2023-06-23T18:35:54.527072+0000","last_undegraded":"2023-06-23T18:35:54.527072+0000","last_fullsized":"2023-06-23T18:35:54.527072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:30.042205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527690+0000","last_change":"2023-06-23T18:35:53.536630+0000","last_active":"2023-06-23T18:35:54.527690+0000","last_peered":"2023-06-23T18:35:54.527690+0000","last_clean":"2023-06-23T18:35:54.527690+0000","last_became_active":"2023-06-23T18:35:53.536298+0000","last_became_peered":"2023-06-23T18:35:53.536298+0000","last_unstale":"2023-06-23T18:35:54.527690+0000","last_undegraded":"2023-06-23T18:35:54.527690+0000","last_fullsized":"2023-06-23T18:35:54.527690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:16.869108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287850+0000","last_change":"2023-06-23T18:35:53.126505+0000","last_active":"2023-06-23T18:35:55.287850+0000","last_peered":"2023-06-23T18:35:55.287850+0000","last_clean":"2023-06-23T18:35:55.287850+0000","last_became_active":"2023-06-23T18:35:53.126226+0000","last_became_peered":"2023-06-23T18:35:53.126226+0000","last_unstale":"2023-06-23T18:35:55.287850+0000","last_undegraded":"2023-06-23T18:35:55.287850+0000","last_fullsized":"2023-06-23T18:35:55.287850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:40.964533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287939+0000","last_change":"2023-06-23T18:35:53.533402+0000","last_active":"2023-06-23T18:35:55.287939+0000","last_peered":"2023-06-23T18:35:55.287939+0000","last_clean":"2023-06-23T18:35:55.287939+0000","last_became_active":"2023-06-23T18:35:53.533163+0000","last_became_peered":"2023-06-23T18:35:53.533163+0000","last_unstale":"2023-06-23T18:35:55.287939+0000","last_undegraded":"2023-06-23T18:35:55.287939+0000","last_fullsized":"2023-06-23T18:35:55.287939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:01:29.821609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526906+0000","last_change":"2023-06-23T18:35:53.123684+0000","last_active":"2023-06-23T18:35:54.526906+0000","last_peered":"2023-06-23T18:35:54.526906+0000","last_clean":"2023-06-23T18:35:54.526906+0000","last_became_active":"2023-06-23T18:35:53.123369+0000","last_became_peered":"2023-06-23T18:35:53.123369+0000","last_unstale":"2023-06-23T18:35:54.526906+0000","last_undegraded":"2023-06-23T18:35:54.526906+0000","last_fullsized":"2023-06-23T18:35:54.526906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:18.604812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525869+0000","last_change":"2023-06-23T18:35:53.534162+0000","last_active":"2023-06-23T18:35:54.525869+0000","last_peered":"2023-06-23T18:35:54.525869+0000","last_clean":"2023-06-23T18:35:54.525869+0000","last_became_active":"2023-06-23T18:35:53.533853+0000","last_became_peered":"2023-06-23T18:35:53.533853+0000","last_unstale":"2023-06-23T18:35:54.525869+0000","last_undegraded":"2023-06-23T18:35:54.525869+0000","last_fullsized":"2023-06-23T18:35:54.525869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:30.583311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527209+0000","last_change":"2023-06-23T18:35:53.123532+0000","last_active":"2023-06-23T18:35:54.527209+0000","last_peered":"2023-06-23T18:35:54.527209+0000","last_clean":"2023-06-23T18:35:54.527209+0000","last_became_active":"2023-06-23T18:35:53.123228+0000","last_became_peered":"2023-06-23T18:35:53.123228+0000","last_unstale":"2023-06-23T18:35:54.527209+0000","last_undegraded":"2023-06-23T18:35:54.527209+0000","last_fullsized":"2023-06-23T18:35:54.527209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:29.581544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527107+0000","last_change":"2023-06-23T18:35:53.533902+0000","last_active":"2023-06-23T18:35:54.527107+0000","last_peered":"2023-06-23T18:35:54.527107+0000","last_clean":"2023-06-23T18:35:54.527107+0000","last_became_active":"2023-06-23T18:35:53.533563+0000","last_became_peered":"2023-06-23T18:35:53.533563+0000","last_unstale":"2023-06-23T18:35:54.527107+0000","last_undegraded":"2023-06-23T18:35:54.527107+0000","last_fullsized":"2023-06-23T18:35:54.527107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:55:12.340901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288866+0000","last_change":"2023-06-23T18:35:53.126195+0000","last_active":"2023-06-23T18:35:55.288866+0000","last_peered":"2023-06-23T18:35:55.288866+0000","last_clean":"2023-06-23T18:35:55.288866+0000","last_became_active":"2023-06-23T18:35:53.125886+0000","last_became_peered":"2023-06-23T18:35:53.125886+0000","last_unstale":"2023-06-23T18:35:55.288866+0000","last_undegraded":"2023-06-23T18:35:55.288866+0000","last_fullsized":"2023-06-23T18:35:55.288866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:57.730157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288616+0000","last_change":"2023-06-23T18:35:53.534747+0000","last_active":"2023-06-23T18:35:55.288616+0000","last_peered":"2023-06-23T18:35:55.288616+0000","last_clean":"2023-06-23T18:35:55.288616+0000","last_became_active":"2023-06-23T18:35:53.534495+0000","last_became_peered":"2023-06-23T18:35:53.534495+0000","last_unstale":"2023-06-23T18:35:55.288616+0000","last_undegraded":"2023-06-23T18:35:55.288616+0000","last_fullsized":"2023-06-23T18:35:55.288616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:30.583311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288416+0000","last_change":"2023-06-23T18:35:53.130647+0000","last_active":"2023-06-23T18:35:55.288416+0000","last_peered":"2023-06-23T18:35:55.288416+0000","last_clean":"2023-06-23T18:35:55.288416+0000","last_became_active":"2023-06-23T18:35:53.129738+0000","last_became_peered":"2023-06-23T18:35:53.129738+0000","last_unstale":"2023-06-23T18:35:55.288416+0000","last_undegraded":"2023-06-23T18:35:55.288416+0000","last_fullsized":"2023-06-23T18:35:55.288416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:16.518353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525676+0000","last_change":"2023-06-23T18:35:53.540648+0000","last_active":"2023-06-23T18:35:54.525676+0000","last_peered":"2023-06-23T18:35:54.525676+0000","last_clean":"2023-06-23T18:35:54.525676+0000","last_became_active":"2023-06-23T18:35:53.540380+0000","last_became_peered":"2023-06-23T18:35:53.540380+0000","last_unstale":"2023-06-23T18:35:54.525676+0000","last_undegraded":"2023-06-23T18:35:54.525676+0000","last_fullsized":"2023-06-23T18:35:54.525676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257206+0000","last_change":"2023-06-23T18:35:53.539029+0000","last_active":"2023-06-23T18:35:55.257206+0000","last_peered":"2023-06-23T18:35:55.257206+0000","last_clean":"2023-06-23T18:35:55.257206+0000","last_became_active":"2023-06-23T18:35:53.538640+0000","last_became_peered":"2023-06-23T18:35:53.538640+0000","last_unstale":"2023-06-23T18:35:55.257206+0000","last_undegraded":"2023-06-23T18:35:55.257206+0000","last_fullsized":"2023-06-23T18:35:55.257206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:52.658179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288459+0000","last_change":"2023-06-23T18:35:53.541022+0000","last_active":"2023-06-23T18:35:55.288459+0000","last_peered":"2023-06-23T18:35:55.288459+0000","last_clean":"2023-06-23T18:35:55.288459+0000","last_became_active":"2023-06-23T18:35:53.540744+0000","last_became_peered":"2023-06-23T18:35:53.540744+0000","last_unstale":"2023-06-23T18:35:55.288459+0000","last_undegraded":"2023-06-23T18:35:55.288459+0000","last_fullsized":"2023-06-23T18:35:55.288459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:31.043972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526923+0000","last_change":"2023-06-23T18:35:53.540406+0000","last_active":"2023-06-23T18:35:54.526923+0000","last_peered":"2023-06-23T18:35:54.526923+0000","last_clean":"2023-06-23T18:35:54.526923+0000","last_became_active":"2023-06-23T18:35:53.540125+0000","last_became_peered":"2023-06-23T18:35:53.540125+0000","last_unstale":"2023-06-23T18:35:54.526923+0000","last_undegraded":"2023-06-23T18:35:54.526923+0000","last_fullsized":"2023-06-23T18:35:54.526923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:45:17.669839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288781+0000","last_change":"2023-06-23T18:35:53.542899+0000","last_active":"2023-06-23T18:35:55.288781+0000","last_peered":"2023-06-23T18:35:55.288781+0000","last_clean":"2023-06-23T18:35:55.288781+0000","last_became_active":"2023-06-23T18:35:53.541798+0000","last_became_peered":"2023-06-23T18:35:53.541798+0000","last_unstale":"2023-06-23T18:35:55.288781+0000","last_undegraded":"2023-06-23T18:35:55.288781+0000","last_fullsized":"2023-06-23T18:35:55.288781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:30.349972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527379+0000","last_change":"2023-06-23T18:35:53.532923+0000","last_active":"2023-06-23T18:35:54.527379+0000","last_peered":"2023-06-23T18:35:54.527379+0000","last_clean":"2023-06-23T18:35:54.527379+0000","last_became_active":"2023-06-23T18:35:53.532674+0000","last_became_peered":"2023-06-23T18:35:53.532674+0000","last_unstale":"2023-06-23T18:35:54.527379+0000","last_undegraded":"2023-06-23T18:35:54.527379+0000","last_fullsized":"2023-06-23T18:35:54.527379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:17.016870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":823296,"data_stored":665215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":74428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":798720,"data_stored":664796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":73649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":200704,"data_stored":73067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":196608,"data_stored":67865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":76048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":75080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:35:57.614 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:35:57.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.762+0000 7f1ab6c6b700 1 Processor -- start 2023-06-23T18:35:57.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.762+0000 7f1ab6c6b700 1 -- start start 2023-06-23T18:35:57.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.762+0000 7f1ab6c6b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1ab00cec00 0x7f1ab00b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.762+0000 7f1ab6c6b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ab015a750 0x7f1ab00b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.762+0000 7f1ab6c6b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab00b1d40 con 0x7f1ab015a380 2023-06-23T18:35:57.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.762+0000 7f1ab6c6b700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab00b1a90 con 0x7f1ab00cec00 2023-06-23T18:35:57.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.762+0000 7f1ab6c6b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab00b2020 con 0x7f1ab015a750 2023-06-23T18:35:57.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.763+0000 7f1aaffff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1ab00cec00 0x7f1ab00b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.763+0000 7f1ab5208700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ab015a750 0x7f1ab00b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.763+0000 7f1ab4a07700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f1ab015a380 0x7f1ab00ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:50472/0 (socket says 172.21.15.79:50472) 2023-06-23T18:35:57.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.763+0000 7f1ab4a07700 1 -- 172.21.15.79:0/2377959531 learned_addr learned my addr 172.21.15.79:0/2377959531 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:57.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.763+0000 7f1aaffff700 1 -- 172.21.15.79:0/2377959531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ab015a750 msgr2=0x7f1ab00b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:57.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.763+0000 7f1aaffff700 1 --2- 172.21.15.79:0/2377959531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ab015a750 0x7f1ab00b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.763+0000 7f1aaffff700 1 -- 172.21.15.79:0/2377959531 >> v1:172.21.15.155:6789/0 conn(0x7f1ab015a380 legacy=0x7f1ab00ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.763+0000 7f1aaffff700 1 -- 172.21.15.79:0/2377959531 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab00ad650 con 0x7f1ab00cec00 2023-06-23T18:35:57.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.763+0000 7f1ab5208700 1 --2- 172.21.15.79:0/2377959531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ab015a750 0x7f1ab00b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:57.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.763+0000 7f1aaffff700 1 --2- 172.21.15.79:0/2377959531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab00cec00 0x7f1ab00b0000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009840 tx=0x7f1a9c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d351ad9661739a41 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:57.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.764+0000 7f1aaf7fe700 1 -- 172.21.15.79:0/2377959531 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c011830 con 0x7f1ab00cec00 2023-06-23T18:35:57.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.764+0000 7f1aaf7fe700 1 -- 172.21.15.79:0/2377959531 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c011990 con 0x7f1ab00cec00 2023-06-23T18:35:57.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.764+0000 7f1aaf7fe700 1 -- 172.21.15.79:0/2377959531 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c011d20 con 0x7f1ab00cec00 2023-06-23T18:35:57.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.764+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/2377959531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab00cec00 msgr2=0x7f1ab00b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.764+0000 7f1ab6c6b700 1 --2- 172.21.15.79:0/2377959531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab00cec00 0x7f1ab00b0000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009840 tx=0x7f1a9c009b50 comp rx=0 tx=0).stop 2023-06-23T18:35:57.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.764+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/2377959531 shutdown_connections 2023-06-23T18:35:57.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.764+0000 7f1ab6c6b700 1 --2- 172.21.15.79:0/2377959531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ab015a750 0x7f1ab00b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.764+0000 7f1ab6c6b700 1 --2- 172.21.15.79:0/2377959531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab00cec00 0x7f1ab00b0000 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.764+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/2377959531 >> 172.21.15.79:0/2377959531 conn(0x7f1ab0089fc0 msgr2=0x7f1ab0099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:57.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.764+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/2377959531 shutdown_connections 2023-06-23T18:35:57.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.764+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/2377959531 wait complete. 2023-06-23T18:35:57.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.765+0000 7f1ab6c6b700 1 Processor -- start 2023-06-23T18:35:57.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.765+0000 7f1ab6c6b700 1 -- start start 2023-06-23T18:35:57.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.766+0000 7f1ab6c6b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ab00cec00 0x7f1ab00b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.766+0000 7f1ab6c6b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab015a380 0x7f1ab015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.766+0000 7f1ab6c6b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ab015a750 0x7f1ab01505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.766+0000 7f1ab6c6b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab015d340 con 0x7f1ab015a750 2023-06-23T18:35:57.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.766+0000 7f1ab6c6b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab015c6c0 con 0x7f1ab015a380 2023-06-23T18:35:57.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.766+0000 7f1ab6c6b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab015c9a0 con 0x7f1ab00cec00 2023-06-23T18:35:57.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.766+0000 7f1ab5208700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ab015a750 0x7f1ab01505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.766+0000 7f1ab5208700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ab015a750 0x7f1ab01505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39168/0 (socket says 172.21.15.79:39168) 2023-06-23T18:35:57.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.766+0000 7f1ab5208700 1 -- 172.21.15.79:0/164914855 learned_addr learned my addr 172.21.15.79:0/164914855 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:57.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.766+0000 7f1aaffff700 1 --2- 172.21.15.79:0/164914855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab015a380 0x7f1ab015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.767+0000 7f1ab4a07700 1 --2- 172.21.15.79:0/164914855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ab00cec00 0x7f1ab00b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.767+0000 7f1ab5208700 1 -- 172.21.15.79:0/164914855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ab00cec00 msgr2=0x7f1ab00b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.767+0000 7f1ab5208700 1 --2- 172.21.15.79:0/164914855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ab00cec00 0x7f1ab00b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.767+0000 7f1ab5208700 1 -- 172.21.15.79:0/164914855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab015a380 msgr2=0x7f1ab015bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.767+0000 7f1ab5208700 1 --2- 172.21.15.79:0/164914855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab015a380 0x7f1ab015bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.767+0000 7f1ab5208700 1 -- 172.21.15.79:0/164914855 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa8009a70 con 0x7f1ab015a750 2023-06-23T18:35:57.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.767+0000 7f1ab5208700 1 --2- 172.21.15.79:0/164914855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ab015a750 0x7f1ab01505c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800ef90 tx=0x7f1aa800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:57.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.767+0000 7f1aad7fa700 1 -- 172.21.15.79:0/164914855 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa800cc90 con 0x7f1ab015a750 2023-06-23T18:35:57.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.767+0000 7f1aad7fa700 1 -- 172.21.15.79:0/164914855 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa800f040 con 0x7f1ab015a750 2023-06-23T18:35:57.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.767+0000 7f1aad7fa700 1 -- 172.21.15.79:0/164914855 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80146c0 con 0x7f1ab015a750 2023-06-23T18:35:57.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.767+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/164914855 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9c0094a0 con 0x7f1ab015a750 2023-06-23T18:35:57.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.767+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/164914855 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab015cf60 con 0x7f1ab015a750 2023-06-23T18:35:57.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.768+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/164914855 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab004e240 con 0x7f1ab015a750 2023-06-23T18:35:57.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.768+0000 7f1aad7fa700 1 -- 172.21.15.79:0/164914855 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1aa8014820 con 0x7f1ab015a750 2023-06-23T18:35:57.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.769+0000 7f1aad7fa700 1 --2- 172.21.15.79:0/164914855 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1a9406ace0 0x7f1a9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:57.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.769+0000 7f1aad7fa700 1 -- 172.21.15.79:0/164914855 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1aa808a010 con 0x7f1ab015a750 2023-06-23T18:35:57.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.769+0000 7f1ab4a07700 1 --2- 172.21.15.79:0/164914855 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1a9406ace0 0x7f1a9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:57.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.769+0000 7f1ab4a07700 1 --2- 172.21.15.79:0/164914855 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1a9406ace0 0x7f1a9406d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4005910 tx=0x7f1aa4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:57.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.773+0000 7f1aad7fa700 1 -- 172.21.15.79:0/164914855 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1aa8055150 con 0x7f1ab015a750 2023-06-23T18:35:57.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.955+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/164914855 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ab00ad930 con 0x7f1a9406ace0 2023-06-23T18:35:57.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.962+0000 7f1aad7fa700 1 -- 172.21.15.79:0/164914855 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287007 (secure 0 0 0) 0x7f1aa4001570 con 0x7f1a9406ace0 2023-06-23T18:35:57.971 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:57.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.966+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/164914855 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1a9406ace0 msgr2=0x7f1a9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.966+0000 7f1ab6c6b700 1 --2- 172.21.15.79:0/164914855 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1a9406ace0 0x7f1a9406d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4005910 tx=0x7f1aa4009040 comp rx=0 tx=0).stop 2023-06-23T18:35:57.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.966+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/164914855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ab015a750 msgr2=0x7f1ab01505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:57.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.966+0000 7f1ab6c6b700 1 --2- 172.21.15.79:0/164914855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ab015a750 0x7f1ab01505c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800ef90 tx=0x7f1aa800c3a0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.966+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/164914855 shutdown_connections 2023-06-23T18:35:57.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.966+0000 7f1ab6c6b700 1 --2- 172.21.15.79:0/164914855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ab015a750 0x7f1ab01505c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.966+0000 7f1ab6c6b700 1 --2- 172.21.15.79:0/164914855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ab00cec00 0x7f1ab00b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.966+0000 7f1ab6c6b700 1 --2- 172.21.15.79:0/164914855 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1a9406ace0 0x7f1a9406d190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.966+0000 7f1ab6c6b700 1 --2- 172.21.15.79:0/164914855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab015a380 0x7f1ab015bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:57.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.966+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/164914855 >> 172.21.15.79:0/164914855 conn(0x7f1ab0089fc0 msgr2=0x7f1ab006fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:57.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.966+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/164914855 shutdown_connections 2023-06-23T18:35:57.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:57.966+0000 7f1ab6c6b700 1 -- 172.21.15.79:0/164914855 wait complete. 2023-06-23T18:35:57.974 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-23T18:35:57.983 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-06-23T18:35:56.695869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218416,"kb_used_data":2800,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756304,"statfs":{"total":773094113280,"available":772870455296,"internally_reserved":0,"allocated":2867200,"data_stored":1770148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.586464"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287219+0000","last_change":"2023-06-23T18:35:53.534999+0000","last_active":"2023-06-23T18:35:55.287219+0000","last_peered":"2023-06-23T18:35:55.287219+0000","last_clean":"2023-06-23T18:35:55.287219+0000","last_became_active":"2023-06-23T18:35:53.534775+0000","last_became_peered":"2023-06-23T18:35:53.534775+0000","last_unstale":"2023-06-23T18:35:55.287219+0000","last_undegraded":"2023-06-23T18:35:55.287219+0000","last_fullsized":"2023-06-23T18:35:55.287219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:15:42.949792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527064+0000","last_change":"2023-06-23T18:35:53.537237+0000","last_active":"2023-06-23T18:35:54.527064+0000","last_peered":"2023-06-23T18:35:54.527064+0000","last_clean":"2023-06-23T18:35:54.527064+0000","last_became_active":"2023-06-23T18:35:53.536900+0000","last_became_peered":"2023-06-23T18:35:53.536900+0000","last_unstale":"2023-06-23T18:35:54.527064+0000","last_undegraded":"2023-06-23T18:35:54.527064+0000","last_fullsized":"2023-06-23T18:35:54.527064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:17.520121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526243+0000","last_change":"2023-06-23T18:35:53.527037+0000","last_active":"2023-06-23T18:35:54.526243+0000","last_peered":"2023-06-23T18:35:54.526243+0000","last_clean":"2023-06-23T18:35:54.526243+0000","last_became_active":"2023-06-23T18:35:53.526655+0000","last_became_peered":"2023-06-23T18:35:53.526655+0000","last_unstale":"2023-06-23T18:35:54.526243+0000","last_undegraded":"2023-06-23T18:35:54.526243+0000","last_fullsized":"2023-06-23T18:35:54.526243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:30.349972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257396+0000","last_change":"2023-06-23T18:35:53.540867+0000","last_active":"2023-06-23T18:35:55.257396+0000","last_peered":"2023-06-23T18:35:55.257396+0000","last_clean":"2023-06-23T18:35:55.257396+0000","last_became_active":"2023-06-23T18:35:53.540572+0000","last_became_peered":"2023-06-23T18:35:53.540572+0000","last_unstale":"2023-06-23T18:35:55.257396+0000","last_undegraded":"2023-06-23T18:35:55.257396+0000","last_fullsized":"2023-06-23T18:35:55.257396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:28.977806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287682+0000","last_change":"2023-06-23T18:35:53.535027+0000","last_active":"2023-06-23T18:35:55.287682+0000","last_peered":"2023-06-23T18:35:55.287682+0000","last_clean":"2023-06-23T18:35:55.287682+0000","last_became_active":"2023-06-23T18:35:53.534793+0000","last_became_peered":"2023-06-23T18:35:53.534793+0000","last_unstale":"2023-06-23T18:35:55.287682+0000","last_undegraded":"2023-06-23T18:35:55.287682+0000","last_fullsized":"2023-06-23T18:35:55.287682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:17.016870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526443+0000","last_change":"2023-06-23T18:35:53.524551+0000","last_active":"2023-06-23T18:35:54.526443+0000","last_peered":"2023-06-23T18:35:54.526443+0000","last_clean":"2023-06-23T18:35:54.526443+0000","last_became_active":"2023-06-23T18:35:53.524271+0000","last_became_peered":"2023-06-23T18:35:53.524271+0000","last_unstale":"2023-06-23T18:35:54.526443+0000","last_undegraded":"2023-06-23T18:35:54.526443+0000","last_fullsized":"2023-06-23T18:35:54.526443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:08:04.256786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793931+0000","last_change":"2023-06-23T18:35:53.540542+0000","last_active":"2023-06-23T18:35:54.793931+0000","last_peered":"2023-06-23T18:35:54.793931+0000","last_clean":"2023-06-23T18:35:54.793931+0000","last_became_active":"2023-06-23T18:35:53.539978+0000","last_became_peered":"2023-06-23T18:35:53.539978+0000","last_unstale":"2023-06-23T18:35:54.793931+0000","last_undegraded":"2023-06-23T18:35:54.793931+0000","last_fullsized":"2023-06-23T18:35:54.793931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:57:46.218466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525592+0000","last_change":"2023-06-23T18:35:53.533541+0000","last_active":"2023-06-23T18:35:54.525592+0000","last_peered":"2023-06-23T18:35:54.525592+0000","last_clean":"2023-06-23T18:35:54.525592+0000","last_became_active":"2023-06-23T18:35:53.531085+0000","last_became_peered":"2023-06-23T18:35:53.531085+0000","last_unstale":"2023-06-23T18:35:54.525592+0000","last_undegraded":"2023-06-23T18:35:54.525592+0000","last_fullsized":"2023-06-23T18:35:54.525592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:52.658179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288165+0000","last_change":"2023-06-23T18:35:53.534716+0000","last_active":"2023-06-23T18:35:55.288165+0000","last_peered":"2023-06-23T18:35:55.288165+0000","last_clean":"2023-06-23T18:35:55.288165+0000","last_became_active":"2023-06-23T18:35:53.534450+0000","last_became_peered":"2023-06-23T18:35:53.534450+0000","last_unstale":"2023-06-23T18:35:55.288165+0000","last_undegraded":"2023-06-23T18:35:55.288165+0000","last_fullsized":"2023-06-23T18:35:55.288165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527020+0000","last_change":"2023-06-23T18:35:53.533580+0000","last_active":"2023-06-23T18:35:54.527020+0000","last_peered":"2023-06-23T18:35:54.527020+0000","last_clean":"2023-06-23T18:35:54.527020+0000","last_became_active":"2023-06-23T18:35:53.533273+0000","last_became_peered":"2023-06-23T18:35:53.533273+0000","last_unstale":"2023-06-23T18:35:54.527020+0000","last_undegraded":"2023-06-23T18:35:54.527020+0000","last_fullsized":"2023-06-23T18:35:54.527020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526443+0000","last_change":"2023-06-23T18:35:53.537638+0000","last_active":"2023-06-23T18:35:54.526443+0000","last_peered":"2023-06-23T18:35:54.526443+0000","last_clean":"2023-06-23T18:35:54.526443+0000","last_became_active":"2023-06-23T18:35:53.537351+0000","last_became_peered":"2023-06-23T18:35:53.537351+0000","last_unstale":"2023-06-23T18:35:54.526443+0000","last_undegraded":"2023-06-23T18:35:54.526443+0000","last_fullsized":"2023-06-23T18:35:54.526443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:25:31.659636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288045+0000","last_change":"2023-06-23T18:35:53.530072+0000","last_active":"2023-06-23T18:35:55.288045+0000","last_peered":"2023-06-23T18:35:55.288045+0000","last_clean":"2023-06-23T18:35:55.288045+0000","last_became_active":"2023-06-23T18:35:53.529676+0000","last_became_peered":"2023-06-23T18:35:53.529676+0000","last_unstale":"2023-06-23T18:35:55.288045+0000","last_undegraded":"2023-06-23T18:35:55.288045+0000","last_fullsized":"2023-06-23T18:35:55.288045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:04:21.490439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287613+0000","last_change":"2023-06-23T18:35:53.535447+0000","last_active":"2023-06-23T18:35:55.287613+0000","last_peered":"2023-06-23T18:35:55.287613+0000","last_clean":"2023-06-23T18:35:55.287613+0000","last_became_active":"2023-06-23T18:35:53.535211+0000","last_became_peered":"2023-06-23T18:35:53.535211+0000","last_unstale":"2023-06-23T18:35:55.287613+0000","last_undegraded":"2023-06-23T18:35:55.287613+0000","last_fullsized":"2023-06-23T18:35:55.287613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:29:59.349736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526832+0000","last_change":"2023-06-23T18:35:53.530753+0000","last_active":"2023-06-23T18:35:54.526832+0000","last_peered":"2023-06-23T18:35:54.526832+0000","last_clean":"2023-06-23T18:35:54.526832+0000","last_became_active":"2023-06-23T18:35:53.530428+0000","last_became_peered":"2023-06-23T18:35:53.530428+0000","last_unstale":"2023-06-23T18:35:54.526832+0000","last_undegraded":"2023-06-23T18:35:54.526832+0000","last_fullsized":"2023-06-23T18:35:54.526832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:00.677969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794039+0000","last_change":"2023-06-23T18:35:53.540712+0000","last_active":"2023-06-23T18:35:54.794039+0000","last_peered":"2023-06-23T18:35:54.794039+0000","last_clean":"2023-06-23T18:35:54.794039+0000","last_became_active":"2023-06-23T18:35:53.540284+0000","last_became_peered":"2023-06-23T18:35:53.540284+0000","last_unstale":"2023-06-23T18:35:54.794039+0000","last_undegraded":"2023-06-23T18:35:54.794039+0000","last_fullsized":"2023-06-23T18:35:54.794039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:12.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525689+0000","last_change":"2023-06-23T18:35:53.524624+0000","last_active":"2023-06-23T18:35:54.525689+0000","last_peered":"2023-06-23T18:35:54.525689+0000","last_clean":"2023-06-23T18:35:54.525689+0000","last_became_active":"2023-06-23T18:35:53.524336+0000","last_became_peered":"2023-06-23T18:35:53.524336+0000","last_unstale":"2023-06-23T18:35:54.525689+0000","last_undegraded":"2023-06-23T18:35:54.525689+0000","last_fullsized":"2023-06-23T18:35:54.525689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:16.869108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288289+0000","last_change":"2023-06-23T18:35:53.534752+0000","last_active":"2023-06-23T18:35:55.288289+0000","last_peered":"2023-06-23T18:35:55.288289+0000","last_clean":"2023-06-23T18:35:55.288289+0000","last_became_active":"2023-06-23T18:35:53.534491+0000","last_became_peered":"2023-06-23T18:35:53.534491+0000","last_unstale":"2023-06-23T18:35:55.288289+0000","last_undegraded":"2023-06-23T18:35:55.288289+0000","last_fullsized":"2023-06-23T18:35:55.288289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:16.869108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526631+0000","last_change":"2023-06-23T18:35:53.530226+0000","last_active":"2023-06-23T18:35:54.526631+0000","last_peered":"2023-06-23T18:35:54.526631+0000","last_clean":"2023-06-23T18:35:54.526631+0000","last_became_active":"2023-06-23T18:35:53.529956+0000","last_became_peered":"2023-06-23T18:35:53.529956+0000","last_unstale":"2023-06-23T18:35:54.526631+0000","last_undegraded":"2023-06-23T18:35:54.526631+0000","last_fullsized":"2023-06-23T18:35:54.526631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:12.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287300+0000","last_change":"2023-06-23T18:35:53.540646+0000","last_active":"2023-06-23T18:35:55.287300+0000","last_peered":"2023-06-23T18:35:55.287300+0000","last_clean":"2023-06-23T18:35:55.287300+0000","last_became_active":"2023-06-23T18:35:53.540370+0000","last_became_peered":"2023-06-23T18:35:53.540370+0000","last_unstale":"2023-06-23T18:35:55.287300+0000","last_undegraded":"2023-06-23T18:35:55.287300+0000","last_fullsized":"2023-06-23T18:35:55.287300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:38:52.875168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526208+0000","last_change":"2023-06-23T18:35:53.539590+0000","last_active":"2023-06-23T18:35:54.526208+0000","last_peered":"2023-06-23T18:35:54.526208+0000","last_clean":"2023-06-23T18:35:54.526208+0000","last_became_active":"2023-06-23T18:35:53.539298+0000","last_became_peered":"2023-06-23T18:35:53.539298+0000","last_unstale":"2023-06-23T18:35:54.526208+0000","last_undegraded":"2023-06-23T18:35:54.526208+0000","last_fullsized":"2023-06-23T18:35:54.526208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:54:07.361405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794125+0000","last_change":"2023-06-23T18:35:53.543302+0000","last_active":"2023-06-23T18:35:54.794125+0000","last_peered":"2023-06-23T18:35:54.794125+0000","last_clean":"2023-06-23T18:35:54.794125+0000","last_became_active":"2023-06-23T18:35:53.542901+0000","last_became_peered":"2023-06-23T18:35:53.542901+0000","last_unstale":"2023-06-23T18:35:54.794125+0000","last_undegraded":"2023-06-23T18:35:54.794125+0000","last_fullsized":"2023-06-23T18:35:54.794125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:26:44.849536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287754+0000","last_change":"2023-06-23T18:35:53.525002+0000","last_active":"2023-06-23T18:35:55.287754+0000","last_peered":"2023-06-23T18:35:55.287754+0000","last_clean":"2023-06-23T18:35:55.287754+0000","last_became_active":"2023-06-23T18:35:53.524694+0000","last_became_peered":"2023-06-23T18:35:53.524694+0000","last_unstale":"2023-06-23T18:35:55.287754+0000","last_undegraded":"2023-06-23T18:35:55.287754+0000","last_fullsized":"2023-06-23T18:35:55.287754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:28.977806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288309+0000","last_change":"2023-06-23T18:35:53.532165+0000","last_active":"2023-06-23T18:35:55.288309+0000","last_peered":"2023-06-23T18:35:55.288309+0000","last_clean":"2023-06-23T18:35:55.288309+0000","last_became_active":"2023-06-23T18:35:53.531883+0000","last_became_peered":"2023-06-23T18:35:53.531883+0000","last_unstale":"2023-06-23T18:35:55.288309+0000","last_undegraded":"2023-06-23T18:35:55.288309+0000","last_fullsized":"2023-06-23T18:35:55.288309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793736+0000","last_change":"2023-06-23T18:35:53.543926+0000","last_active":"2023-06-23T18:35:54.793736+0000","last_peered":"2023-06-23T18:35:54.793736+0000","last_clean":"2023-06-23T18:35:54.793736+0000","last_became_active":"2023-06-23T18:35:53.543692+0000","last_became_peered":"2023-06-23T18:35:53.543692+0000","last_unstale":"2023-06-23T18:35:54.793736+0000","last_undegraded":"2023-06-23T18:35:54.793736+0000","last_fullsized":"2023-06-23T18:35:54.793736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:19:44.074507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527502+0000","last_change":"2023-06-23T18:35:53.539274+0000","last_active":"2023-06-23T18:35:54.527502+0000","last_peered":"2023-06-23T18:35:54.527502+0000","last_clean":"2023-06-23T18:35:54.527502+0000","last_became_active":"2023-06-23T18:35:53.539000+0000","last_became_peered":"2023-06-23T18:35:53.539000+0000","last_unstale":"2023-06-23T18:35:54.527502+0000","last_undegraded":"2023-06-23T18:35:54.527502+0000","last_fullsized":"2023-06-23T18:35:54.527502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:00.677969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527478+0000","last_change":"2023-06-23T18:35:53.532343+0000","last_active":"2023-06-23T18:35:54.527478+0000","last_peered":"2023-06-23T18:35:54.527478+0000","last_clean":"2023-06-23T18:35:54.527478+0000","last_became_active":"2023-06-23T18:35:53.532137+0000","last_became_peered":"2023-06-23T18:35:53.532137+0000","last_unstale":"2023-06-23T18:35:54.527478+0000","last_undegraded":"2023-06-23T18:35:54.527478+0000","last_fullsized":"2023-06-23T18:35:54.527478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:41.966301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526775+0000","last_change":"2023-06-23T18:35:53.541546+0000","last_active":"2023-06-23T18:35:54.526775+0000","last_peered":"2023-06-23T18:35:54.526775+0000","last_clean":"2023-06-23T18:35:54.526775+0000","last_became_active":"2023-06-23T18:35:53.541285+0000","last_became_peered":"2023-06-23T18:35:53.541285+0000","last_unstale":"2023-06-23T18:35:54.526775+0000","last_undegraded":"2023-06-23T18:35:54.526775+0000","last_fullsized":"2023-06-23T18:35:54.526775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:41.966301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527025+0000","last_change":"2023-06-23T18:35:53.130775+0000","last_active":"2023-06-23T18:35:54.527025+0000","last_peered":"2023-06-23T18:35:54.527025+0000","last_clean":"2023-06-23T18:35:54.527025+0000","last_became_active":"2023-06-23T18:35:53.130473+0000","last_became_peered":"2023-06-23T18:35:53.130473+0000","last_unstale":"2023-06-23T18:35:54.527025+0000","last_undegraded":"2023-06-23T18:35:54.527025+0000","last_fullsized":"2023-06-23T18:35:54.527025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:26:43.847769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288817+0000","last_change":"2023-06-23T18:35:53.530941+0000","last_active":"2023-06-23T18:35:55.288817+0000","last_peered":"2023-06-23T18:35:55.288817+0000","last_clean":"2023-06-23T18:35:55.288817+0000","last_became_active":"2023-06-23T18:35:53.530683+0000","last_became_peered":"2023-06-23T18:35:53.530683+0000","last_unstale":"2023-06-23T18:35:55.288817+0000","last_undegraded":"2023-06-23T18:35:55.288817+0000","last_fullsized":"2023-06-23T18:35:55.288817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:19:44.074507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527326+0000","last_change":"2023-06-23T18:35:53.132155+0000","last_active":"2023-06-23T18:35:54.527326+0000","last_peered":"2023-06-23T18:35:54.527326+0000","last_clean":"2023-06-23T18:35:54.527326+0000","last_became_active":"2023-06-23T18:35:53.131785+0000","last_became_peered":"2023-06-23T18:35:53.131785+0000","last_unstale":"2023-06-23T18:35:54.527326+0000","last_undegraded":"2023-06-23T18:35:54.527326+0000","last_fullsized":"2023-06-23T18:35:54.527326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:51.656412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288482+0000","last_change":"2023-06-23T18:35:53.537294+0000","last_active":"2023-06-23T18:35:55.288482+0000","last_peered":"2023-06-23T18:35:55.288482+0000","last_clean":"2023-06-23T18:35:55.288482+0000","last_became_active":"2023-06-23T18:35:53.536977+0000","last_became_peered":"2023-06-23T18:35:53.536977+0000","last_unstale":"2023-06-23T18:35:55.288482+0000","last_undegraded":"2023-06-23T18:35:55.288482+0000","last_fullsized":"2023-06-23T18:35:55.288482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:30.583311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288741+0000","last_change":"2023-06-23T18:35:53.135828+0000","last_active":"2023-06-23T18:35:55.288741+0000","last_peered":"2023-06-23T18:35:55.288741+0000","last_clean":"2023-06-23T18:35:55.288741+0000","last_became_active":"2023-06-23T18:35:53.135453+0000","last_became_peered":"2023-06-23T18:35:53.135453+0000","last_unstale":"2023-06-23T18:35:55.288741+0000","last_undegraded":"2023-06-23T18:35:55.288741+0000","last_fullsized":"2023-06-23T18:35:55.288741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:54:06.359638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257486+0000","last_change":"2023-06-23T18:35:53.535934+0000","last_active":"2023-06-23T18:35:55.257486+0000","last_peered":"2023-06-23T18:35:55.257486+0000","last_clean":"2023-06-23T18:35:55.257486+0000","last_became_active":"2023-06-23T18:35:53.535505+0000","last_became_peered":"2023-06-23T18:35:53.535505+0000","last_unstale":"2023-06-23T18:35:55.257486+0000","last_undegraded":"2023-06-23T18:35:55.257486+0000","last_fullsized":"2023-06-23T18:35:55.257486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288360+0000","last_change":"2023-06-23T18:35:53.130045+0000","last_active":"2023-06-23T18:35:55.288360+0000","last_peered":"2023-06-23T18:35:55.288360+0000","last_clean":"2023-06-23T18:35:55.288360+0000","last_became_active":"2023-06-23T18:35:53.129374+0000","last_became_peered":"2023-06-23T18:35:53.129374+0000","last_unstale":"2023-06-23T18:35:55.288360+0000","last_undegraded":"2023-06-23T18:35:55.288360+0000","last_fullsized":"2023-06-23T18:35:55.288360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:30.042205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288533+0000","last_change":"2023-06-23T18:35:53.540785+0000","last_active":"2023-06-23T18:35:55.288533+0000","last_peered":"2023-06-23T18:35:55.288533+0000","last_clean":"2023-06-23T18:35:55.288533+0000","last_became_active":"2023-06-23T18:35:53.540511+0000","last_became_peered":"2023-06-23T18:35:53.540511+0000","last_unstale":"2023-06-23T18:35:55.288533+0000","last_undegraded":"2023-06-23T18:35:55.288533+0000","last_fullsized":"2023-06-23T18:35:55.288533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:17.520121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526768+0000","last_change":"2023-06-23T18:35:53.133079+0000","last_active":"2023-06-23T18:35:54.526768+0000","last_peered":"2023-06-23T18:35:54.526768+0000","last_clean":"2023-06-23T18:35:54.526768+0000","last_became_active":"2023-06-23T18:35:53.132662+0000","last_became_peered":"2023-06-23T18:35:53.132662+0000","last_unstale":"2023-06-23T18:35:54.526768+0000","last_undegraded":"2023-06-23T18:35:54.526768+0000","last_fullsized":"2023-06-23T18:35:54.526768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:06.645556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288606+0000","last_change":"2023-06-23T18:35:53.541521+0000","last_active":"2023-06-23T18:35:55.288606+0000","last_peered":"2023-06-23T18:35:55.288606+0000","last_clean":"2023-06-23T18:35:55.288606+0000","last_became_active":"2023-06-23T18:35:53.541265+0000","last_became_peered":"2023-06-23T18:35:53.541265+0000","last_unstale":"2023-06-23T18:35:55.288606+0000","last_undegraded":"2023-06-23T18:35:55.288606+0000","last_fullsized":"2023-06-23T18:35:55.288606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:39.018447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793951+0000","last_change":"2023-06-23T18:35:53.131904+0000","last_active":"2023-06-23T18:35:54.793951+0000","last_peered":"2023-06-23T18:35:54.793951+0000","last_clean":"2023-06-23T18:35:54.793951+0000","last_became_active":"2023-06-23T18:35:53.131581+0000","last_became_peered":"2023-06-23T18:35:53.131581+0000","last_unstale":"2023-06-23T18:35:54.793951+0000","last_undegraded":"2023-06-23T18:35:54.793951+0000","last_fullsized":"2023-06-23T18:35:54.793951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:25:30.657869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527297+0000","last_change":"2023-06-23T18:35:53.542076+0000","last_active":"2023-06-23T18:35:54.527297+0000","last_peered":"2023-06-23T18:35:54.527297+0000","last_clean":"2023-06-23T18:35:54.527297+0000","last_became_active":"2023-06-23T18:35:53.541814+0000","last_became_peered":"2023-06-23T18:35:53.541814+0000","last_unstale":"2023-06-23T18:35:54.527297+0000","last_undegraded":"2023-06-23T18:35:54.527297+0000","last_fullsized":"2023-06-23T18:35:54.527297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:40:01.768196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526932+0000","last_change":"2023-06-23T18:35:53.131165+0000","last_active":"2023-06-23T18:35:54.526932+0000","last_peered":"2023-06-23T18:35:54.526932+0000","last_clean":"2023-06-23T18:35:54.526932+0000","last_became_active":"2023-06-23T18:35:53.130890+0000","last_became_peered":"2023-06-23T18:35:53.130890+0000","last_unstale":"2023-06-23T18:35:54.526932+0000","last_undegraded":"2023-06-23T18:35:54.526932+0000","last_fullsized":"2023-06-23T18:35:54.526932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:16:18.754209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526252+0000","last_change":"2023-06-23T18:35:53.541719+0000","last_active":"2023-06-23T18:35:54.526252+0000","last_peered":"2023-06-23T18:35:54.526252+0000","last_clean":"2023-06-23T18:35:54.526252+0000","last_became_active":"2023-06-23T18:35:53.541456+0000","last_became_peered":"2023-06-23T18:35:53.541456+0000","last_unstale":"2023-06-23T18:35:54.526252+0000","last_undegraded":"2023-06-23T18:35:54.526252+0000","last_fullsized":"2023-06-23T18:35:54.526252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:30:45.259696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794066+0000","last_change":"2023-06-23T18:35:53.136124+0000","last_active":"2023-06-23T18:35:54.794066+0000","last_peered":"2023-06-23T18:35:54.794066+0000","last_clean":"2023-06-23T18:35:54.794066+0000","last_became_active":"2023-06-23T18:35:53.135736+0000","last_became_peered":"2023-06-23T18:35:53.135736+0000","last_unstale":"2023-06-23T18:35:54.794066+0000","last_undegraded":"2023-06-23T18:35:54.794066+0000","last_fullsized":"2023-06-23T18:35:54.794066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:45:16.668072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.289031+0000","last_change":"2023-06-23T18:35:53.540586+0000","last_active":"2023-06-23T18:35:55.289031+0000","last_peered":"2023-06-23T18:35:55.289031+0000","last_clean":"2023-06-23T18:35:55.289031+0000","last_became_active":"2023-06-23T18:35:53.540355+0000","last_became_peered":"2023-06-23T18:35:53.540355+0000","last_unstale":"2023-06-23T18:35:55.289031+0000","last_undegraded":"2023-06-23T18:35:55.289031+0000","last_fullsized":"2023-06-23T18:35:55.289031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:02:35.233797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287933+0000","last_change":"2023-06-23T18:35:53.133113+0000","last_active":"2023-06-23T18:35:55.287933+0000","last_peered":"2023-06-23T18:35:55.287933+0000","last_clean":"2023-06-23T18:35:55.287933+0000","last_became_active":"2023-06-23T18:35:53.132833+0000","last_became_peered":"2023-06-23T18:35:53.132833+0000","last_unstale":"2023-06-23T18:35:55.287933+0000","last_undegraded":"2023-06-23T18:35:55.287933+0000","last_fullsized":"2023-06-23T18:35:55.287933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:46:59.676202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527016+0000","last_change":"2023-06-23T18:35:53.538309+0000","last_active":"2023-06-23T18:35:54.527016+0000","last_peered":"2023-06-23T18:35:54.527016+0000","last_clean":"2023-06-23T18:35:54.527016+0000","last_became_active":"2023-06-23T18:35:53.538037+0000","last_became_peered":"2023-06-23T18:35:53.538037+0000","last_unstale":"2023-06-23T18:35:54.527016+0000","last_undegraded":"2023-06-23T18:35:54.527016+0000","last_fullsized":"2023-06-23T18:35:54.527016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793844+0000","last_change":"2023-06-23T18:35:53.131958+0000","last_active":"2023-06-23T18:35:54.793844+0000","last_peered":"2023-06-23T18:35:54.793844+0000","last_clean":"2023-06-23T18:35:54.793844+0000","last_became_active":"2023-06-23T18:35:53.131643+0000","last_became_peered":"2023-06-23T18:35:53.131643+0000","last_unstale":"2023-06-23T18:35:54.793844+0000","last_undegraded":"2023-06-23T18:35:54.793844+0000","last_fullsized":"2023-06-23T18:35:54.793844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:57.730157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288109+0000","last_change":"2023-06-23T18:35:53.533318+0000","last_active":"2023-06-23T18:35:55.288109+0000","last_peered":"2023-06-23T18:35:55.288109+0000","last_clean":"2023-06-23T18:35:55.288109+0000","last_became_active":"2023-06-23T18:35:53.533081+0000","last_became_peered":"2023-06-23T18:35:53.533081+0000","last_unstale":"2023-06-23T18:35:55.288109+0000","last_undegraded":"2023-06-23T18:35:55.288109+0000","last_fullsized":"2023-06-23T18:35:55.288109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257090+0000","last_change":"2023-06-23T18:35:53.135982+0000","last_active":"2023-06-23T18:35:55.257090+0000","last_peered":"2023-06-23T18:35:55.257090+0000","last_clean":"2023-06-23T18:35:55.257090+0000","last_became_active":"2023-06-23T18:35:53.135591+0000","last_became_peered":"2023-06-23T18:35:53.135591+0000","last_unstale":"2023-06-23T18:35:55.257090+0000","last_undegraded":"2023-06-23T18:35:55.257090+0000","last_fullsized":"2023-06-23T18:35:55.257090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:40:00.766429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288988+0000","last_change":"2023-06-23T18:35:53.541883+0000","last_active":"2023-06-23T18:35:55.288988+0000","last_peered":"2023-06-23T18:35:55.288988+0000","last_clean":"2023-06-23T18:35:55.288988+0000","last_became_active":"2023-06-23T18:35:53.541630+0000","last_became_peered":"2023-06-23T18:35:53.541630+0000","last_unstale":"2023-06-23T18:35:55.288988+0000","last_undegraded":"2023-06-23T18:35:55.288988+0000","last_fullsized":"2023-06-23T18:35:55.288988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:08:04.256786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793955+0000","last_change":"2023-06-23T18:35:53.132583+0000","last_active":"2023-06-23T18:35:54.793955+0000","last_peered":"2023-06-23T18:35:54.793955+0000","last_clean":"2023-06-23T18:35:54.793955+0000","last_became_active":"2023-06-23T18:35:53.132142+0000","last_became_peered":"2023-06-23T18:35:53.132142+0000","last_unstale":"2023-06-23T18:35:54.793955+0000","last_undegraded":"2023-06-23T18:35:54.793955+0000","last_fullsized":"2023-06-23T18:35:54.793955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:02:34.232029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526009+0000","last_change":"2023-06-23T18:35:53.537549+0000","last_active":"2023-06-23T18:35:54.526009+0000","last_peered":"2023-06-23T18:35:54.526009+0000","last_clean":"2023-06-23T18:35:54.526009+0000","last_became_active":"2023-06-23T18:35:53.537281+0000","last_became_peered":"2023-06-23T18:35:53.537281+0000","last_unstale":"2023-06-23T18:35:54.526009+0000","last_undegraded":"2023-06-23T18:35:54.526009+0000","last_fullsized":"2023-06-23T18:35:54.526009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:17.016870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794202+0000","last_change":"2023-06-23T18:35:53.131954+0000","last_active":"2023-06-23T18:35:54.794202+0000","last_peered":"2023-06-23T18:35:54.794202+0000","last_clean":"2023-06-23T18:35:54.794202+0000","last_became_active":"2023-06-23T18:35:53.131624+0000","last_became_peered":"2023-06-23T18:35:53.131624+0000","last_unstale":"2023-06-23T18:35:54.794202+0000","last_undegraded":"2023-06-23T18:35:54.794202+0000","last_fullsized":"2023-06-23T18:35:54.794202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:16:18.754209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525847+0000","last_change":"2023-06-23T18:35:53.535467+0000","last_active":"2023-06-23T18:35:54.525847+0000","last_peered":"2023-06-23T18:35:54.525847+0000","last_clean":"2023-06-23T18:35:54.525847+0000","last_became_active":"2023-06-23T18:35:53.535187+0000","last_became_peered":"2023-06-23T18:35:53.535187+0000","last_unstale":"2023-06-23T18:35:54.525847+0000","last_undegraded":"2023-06-23T18:35:54.525847+0000","last_fullsized":"2023-06-23T18:35:54.525847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:31.043972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793955+0000","last_change":"2023-06-23T18:35:53.132445+0000","last_active":"2023-06-23T18:35:54.793955+0000","last_peered":"2023-06-23T18:35:54.793955+0000","last_clean":"2023-06-23T18:35:54.793955+0000","last_became_active":"2023-06-23T18:35:53.132108+0000","last_became_peered":"2023-06-23T18:35:53.132108+0000","last_unstale":"2023-06-23T18:35:54.793955+0000","last_undegraded":"2023-06-23T18:35:54.793955+0000","last_fullsized":"2023-06-23T18:35:54.793955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:44:02.521500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257388+0000","last_change":"2023-06-23T18:35:53.539086+0000","last_active":"2023-06-23T18:35:55.257388+0000","last_peered":"2023-06-23T18:35:55.257388+0000","last_clean":"2023-06-23T18:35:55.257388+0000","last_became_active":"2023-06-23T18:35:53.538755+0000","last_became_peered":"2023-06-23T18:35:53.538755+0000","last_unstale":"2023-06-23T18:35:55.257388+0000","last_undegraded":"2023-06-23T18:35:55.257388+0000","last_fullsized":"2023-06-23T18:35:55.257388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:07.647324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527056+0000","last_change":"2023-06-23T18:35:53.128100+0000","last_active":"2023-06-23T18:35:54.527056+0000","last_peered":"2023-06-23T18:35:54.527056+0000","last_clean":"2023-06-23T18:35:54.527056+0000","last_became_active":"2023-06-23T18:35:53.127782+0000","last_became_peered":"2023-06-23T18:35:53.127782+0000","last_unstale":"2023-06-23T18:35:54.527056+0000","last_undegraded":"2023-06-23T18:35:54.527056+0000","last_fullsized":"2023-06-23T18:35:54.527056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:15.867340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287463+0000","last_change":"2023-06-23T18:35:53.524781+0000","last_active":"2023-06-23T18:35:55.287463+0000","last_peered":"2023-06-23T18:35:55.287463+0000","last_clean":"2023-06-23T18:35:55.287463+0000","last_became_active":"2023-06-23T18:35:53.524526+0000","last_became_peered":"2023-06-23T18:35:53.524526+0000","last_unstale":"2023-06-23T18:35:55.287463+0000","last_undegraded":"2023-06-23T18:35:55.287463+0000","last_fullsized":"2023-06-23T18:35:55.287463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:40:01.768196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288801+0000","last_change":"2023-06-23T18:35:53.133550+0000","last_active":"2023-06-23T18:35:55.288801+0000","last_peered":"2023-06-23T18:35:55.288801+0000","last_clean":"2023-06-23T18:35:55.288801+0000","last_became_active":"2023-06-23T18:35:53.133287+0000","last_became_peered":"2023-06-23T18:35:53.133287+0000","last_unstale":"2023-06-23T18:35:55.288801+0000","last_undegraded":"2023-06-23T18:35:55.288801+0000","last_fullsized":"2023-06-23T18:35:55.288801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:30:44.257929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526733+0000","last_change":"2023-06-23T18:35:53.534074+0000","last_active":"2023-06-23T18:35:54.526733+0000","last_peered":"2023-06-23T18:35:54.526733+0000","last_clean":"2023-06-23T18:35:54.526733+0000","last_became_active":"2023-06-23T18:35:53.533789+0000","last_became_peered":"2023-06-23T18:35:53.533789+0000","last_unstale":"2023-06-23T18:35:54.526733+0000","last_undegraded":"2023-06-23T18:35:54.526733+0000","last_fullsized":"2023-06-23T18:35:54.526733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:58.731924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257244+0000","last_change":"2023-06-23T18:35:53.126156+0000","last_active":"2023-06-23T18:35:55.257244+0000","last_peered":"2023-06-23T18:35:55.257244+0000","last_clean":"2023-06-23T18:35:55.257244+0000","last_became_active":"2023-06-23T18:35:53.125640+0000","last_became_peered":"2023-06-23T18:35:53.125640+0000","last_unstale":"2023-06-23T18:35:55.257244+0000","last_undegraded":"2023-06-23T18:35:55.257244+0000","last_fullsized":"2023-06-23T18:35:55.257244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:16.015103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526631+0000","last_change":"2023-06-23T18:35:53.529926+0000","last_active":"2023-06-23T18:35:54.526631+0000","last_peered":"2023-06-23T18:35:54.526631+0000","last_clean":"2023-06-23T18:35:54.526631+0000","last_became_active":"2023-06-23T18:35:53.529619+0000","last_became_peered":"2023-06-23T18:35:53.529619+0000","last_unstale":"2023-06-23T18:35:54.526631+0000","last_undegraded":"2023-06-23T18:35:54.526631+0000","last_fullsized":"2023-06-23T18:35:54.526631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:44:03.523267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526406+0000","last_change":"2023-06-23T18:35:53.134550+0000","last_active":"2023-06-23T18:35:54.526406+0000","last_peered":"2023-06-23T18:35:54.526406+0000","last_clean":"2023-06-23T18:35:54.526406+0000","last_became_active":"2023-06-23T18:35:53.132753+0000","last_became_peered":"2023-06-23T18:35:53.132753+0000","last_unstale":"2023-06-23T18:35:54.526406+0000","last_undegraded":"2023-06-23T18:35:54.526406+0000","last_fullsized":"2023-06-23T18:35:54.526406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:19:43.072740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526469+0000","last_change":"2023-06-23T18:35:53.541907+0000","last_active":"2023-06-23T18:35:54.526469+0000","last_peered":"2023-06-23T18:35:54.526469+0000","last_clean":"2023-06-23T18:35:54.526469+0000","last_became_active":"2023-06-23T18:35:53.541545+0000","last_became_peered":"2023-06-23T18:35:53.541545+0000","last_unstale":"2023-06-23T18:35:54.526469+0000","last_undegraded":"2023-06-23T18:35:54.526469+0000","last_fullsized":"2023-06-23T18:35:54.526469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:02:35.233797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288368+0000","last_change":"2023-06-23T18:35:53.130943+0000","last_active":"2023-06-23T18:35:55.288368+0000","last_peered":"2023-06-23T18:35:55.288368+0000","last_clean":"2023-06-23T18:35:55.288368+0000","last_became_active":"2023-06-23T18:35:53.130544+0000","last_became_peered":"2023-06-23T18:35:53.130544+0000","last_unstale":"2023-06-23T18:35:55.288368+0000","last_undegraded":"2023-06-23T18:35:55.288368+0000","last_fullsized":"2023-06-23T18:35:55.288368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:18.373567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527635+0000","last_change":"2023-06-23T18:35:53.535355+0000","last_active":"2023-06-23T18:35:54.527635+0000","last_peered":"2023-06-23T18:35:54.527635+0000","last_clean":"2023-06-23T18:35:54.527635+0000","last_became_active":"2023-06-23T18:35:53.535087+0000","last_became_peered":"2023-06-23T18:35:53.535087+0000","last_unstale":"2023-06-23T18:35:54.527635+0000","last_undegraded":"2023-06-23T18:35:54.527635+0000","last_fullsized":"2023-06-23T18:35:54.527635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288978+0000","last_change":"2023-06-23T18:35:53.133469+0000","last_active":"2023-06-23T18:35:55.288978+0000","last_peered":"2023-06-23T18:35:55.288978+0000","last_clean":"2023-06-23T18:35:55.288978+0000","last_became_active":"2023-06-23T18:35:53.133211+0000","last_became_peered":"2023-06-23T18:35:53.133211+0000","last_unstale":"2023-06-23T18:35:55.288978+0000","last_undegraded":"2023-06-23T18:35:55.288978+0000","last_fullsized":"2023-06-23T18:35:55.288978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:57:45.216699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526357+0000","last_change":"2023-06-23T18:35:48.920009+0000","last_active":"2023-06-23T18:35:54.526357+0000","last_peered":"2023-06-23T18:35:54.526357+0000","last_clean":"2023-06-23T18:35:54.526357+0000","last_became_active":"2023-06-23T18:35:48.919686+0000","last_became_peered":"2023-06-23T18:35:48.919686+0000","last_unstale":"2023-06-23T18:35:54.526357+0000","last_undegraded":"2023-06-23T18:35:54.526357+0000","last_fullsized":"2023-06-23T18:35:54.526357+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:57:40.062937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288250+0000","last_change":"2023-06-23T18:35:53.538294+0000","last_active":"2023-06-23T18:35:55.288250+0000","last_peered":"2023-06-23T18:35:55.288250+0000","last_clean":"2023-06-23T18:35:55.288250+0000","last_became_active":"2023-06-23T18:35:53.537996+0000","last_became_peered":"2023-06-23T18:35:53.537996+0000","last_unstale":"2023-06-23T18:35:55.288250+0000","last_undegraded":"2023-06-23T18:35:55.288250+0000","last_fullsized":"2023-06-23T18:35:55.288250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288977+0000","last_change":"2023-06-23T18:35:47.962164+0000","last_active":"2023-06-23T18:35:55.288977+0000","last_peered":"2023-06-23T18:35:55.288977+0000","last_clean":"2023-06-23T18:35:55.288977+0000","last_became_active":"2023-06-23T18:35:47.961790+0000","last_became_peered":"2023-06-23T18:35:47.961790+0000","last_unstale":"2023-06-23T18:35:55.288977+0000","last_undegraded":"2023-06-23T18:35:55.288977+0000","last_fullsized":"2023-06-23T18:35:55.288977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:25:25.504108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288539+0000","last_change":"2023-06-23T18:35:53.126850+0000","last_active":"2023-06-23T18:35:55.288539+0000","last_peered":"2023-06-23T18:35:55.288539+0000","last_clean":"2023-06-23T18:35:55.288539+0000","last_became_active":"2023-06-23T18:35:53.126560+0000","last_became_peered":"2023-06-23T18:35:53.126560+0000","last_unstale":"2023-06-23T18:35:55.288539+0000","last_undegraded":"2023-06-23T18:35:55.288539+0000","last_fullsized":"2023-06-23T18:35:55.288539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:38.016680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526607+0000","last_change":"2023-06-23T18:35:53.531624+0000","last_active":"2023-06-23T18:35:54.526607+0000","last_peered":"2023-06-23T18:35:54.526607+0000","last_clean":"2023-06-23T18:35:54.526607+0000","last_became_active":"2023-06-23T18:35:53.531197+0000","last_became_peered":"2023-06-23T18:35:53.531197+0000","last_unstale":"2023-06-23T18:35:54.526607+0000","last_undegraded":"2023-06-23T18:35:54.526607+0000","last_fullsized":"2023-06-23T18:35:54.526607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:17.520121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527435+0000","last_change":"2023-06-23T18:35:47.965867+0000","last_active":"2023-06-23T18:35:54.527435+0000","last_peered":"2023-06-23T18:35:54.527435+0000","last_clean":"2023-06-23T18:35:54.527435+0000","last_became_active":"2023-06-23T18:35:47.965580+0000","last_became_peered":"2023-06-23T18:35:47.965580+0000","last_unstale":"2023-06-23T18:35:54.527435+0000","last_undegraded":"2023-06-23T18:35:54.527435+0000","last_fullsized":"2023-06-23T18:35:54.527435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:01.491795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793869+0000","last_change":"2023-06-23T18:35:47.968534+0000","last_active":"2023-06-23T18:35:54.793869+0000","last_peered":"2023-06-23T18:35:54.793869+0000","last_clean":"2023-06-23T18:35:54.793869+0000","last_became_active":"2023-06-23T18:35:47.967508+0000","last_became_peered":"2023-06-23T18:35:47.967508+0000","last_unstale":"2023-06-23T18:35:54.793869+0000","last_undegraded":"2023-06-23T18:35:54.793869+0000","last_fullsized":"2023-06-23T18:35:54.793869+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:07:58.101257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287523+0000","last_change":"2023-06-23T18:35:53.126632+0000","last_active":"2023-06-23T18:35:55.287523+0000","last_peered":"2023-06-23T18:35:55.287523+0000","last_clean":"2023-06-23T18:35:55.287523+0000","last_became_active":"2023-06-23T18:35:53.126336+0000","last_became_peered":"2023-06-23T18:35:53.126336+0000","last_unstale":"2023-06-23T18:35:55.287523+0000","last_undegraded":"2023-06-23T18:35:55.287523+0000","last_fullsized":"2023-06-23T18:35:55.287523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:51.656412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288240+0000","last_change":"2023-06-23T18:35:53.524885+0000","last_active":"2023-06-23T18:35:55.288240+0000","last_peered":"2023-06-23T18:35:55.288240+0000","last_clean":"2023-06-23T18:35:55.288240+0000","last_became_active":"2023-06-23T18:35:53.524630+0000","last_became_peered":"2023-06-23T18:35:53.524630+0000","last_unstale":"2023-06-23T18:35:55.288240+0000","last_undegraded":"2023-06-23T18:35:55.288240+0000","last_fullsized":"2023-06-23T18:35:55.288240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793589+0000","last_change":"2023-06-23T18:35:48.736407+0000","last_active":"2023-06-23T18:35:54.793589+0000","last_peered":"2023-06-23T18:35:54.793589+0000","last_clean":"2023-06-23T18:35:54.793589+0000","last_became_active":"2023-06-23T18:35:48.735621+0000","last_became_peered":"2023-06-23T18:35:48.735621+0000","last_unstale":"2023-06-23T18:35:54.793589+0000","last_undegraded":"2023-06-23T18:35:54.793589+0000","last_fullsized":"2023-06-23T18:35:54.793589+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:30:39.104167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527092+0000","last_change":"2023-06-23T18:35:53.131817+0000","last_active":"2023-06-23T18:35:54.527092+0000","last_peered":"2023-06-23T18:35:54.527092+0000","last_clean":"2023-06-23T18:35:54.527092+0000","last_became_active":"2023-06-23T18:35:53.131549+0000","last_became_peered":"2023-06-23T18:35:53.131549+0000","last_unstale":"2023-06-23T18:35:54.527092+0000","last_undegraded":"2023-06-23T18:35:54.527092+0000","last_fullsized":"2023-06-23T18:35:54.527092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:04:00.010842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527627+0000","last_change":"2023-06-23T18:35:53.531485+0000","last_active":"2023-06-23T18:35:54.527627+0000","last_peered":"2023-06-23T18:35:54.527627+0000","last_clean":"2023-06-23T18:35:54.527627+0000","last_became_active":"2023-06-23T18:35:53.531276+0000","last_became_peered":"2023-06-23T18:35:53.531276+0000","last_unstale":"2023-06-23T18:35:54.527627+0000","last_undegraded":"2023-06-23T18:35:54.527627+0000","last_fullsized":"2023-06-23T18:35:54.527627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793552+0000","last_change":"2023-06-23T18:35:48.736449+0000","last_active":"2023-06-23T18:35:54.793552+0000","last_peered":"2023-06-23T18:35:54.793552+0000","last_clean":"2023-06-23T18:35:54.793552+0000","last_became_active":"2023-06-23T18:35:48.735664+0000","last_became_peered":"2023-06-23T18:35:48.735664+0000","last_unstale":"2023-06-23T18:35:54.793552+0000","last_undegraded":"2023-06-23T18:35:54.793552+0000","last_fullsized":"2023-06-23T18:35:54.793552+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:24.194443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288399+0000","last_change":"2023-06-23T18:35:53.127210+0000","last_active":"2023-06-23T18:35:55.288399+0000","last_peered":"2023-06-23T18:35:55.288399+0000","last_clean":"2023-06-23T18:35:55.288399+0000","last_became_active":"2023-06-23T18:35:53.126894+0000","last_became_peered":"2023-06-23T18:35:53.126894+0000","last_unstale":"2023-06-23T18:35:55.288399+0000","last_undegraded":"2023-06-23T18:35:55.288399+0000","last_fullsized":"2023-06-23T18:35:55.288399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:15.867340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257494+0000","last_change":"2023-06-23T18:35:53.539026+0000","last_active":"2023-06-23T18:35:55.257494+0000","last_peered":"2023-06-23T18:35:55.257494+0000","last_clean":"2023-06-23T18:35:55.257494+0000","last_became_active":"2023-06-23T18:35:53.538711+0000","last_became_peered":"2023-06-23T18:35:53.538711+0000","last_unstale":"2023-06-23T18:35:55.257494+0000","last_undegraded":"2023-06-23T18:35:55.257494+0000","last_fullsized":"2023-06-23T18:35:55.257494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:00.677969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526508+0000","last_change":"2023-06-23T18:35:48.735713+0000","last_active":"2023-06-23T18:35:54.526508+0000","last_peered":"2023-06-23T18:35:54.526508+0000","last_clean":"2023-06-23T18:35:54.526508+0000","last_became_active":"2023-06-23T18:35:48.735236+0000","last_became_peered":"2023-06-23T18:35:48.735236+0000","last_unstale":"2023-06-23T18:35:54.526508+0000","last_undegraded":"2023-06-23T18:35:54.526508+0000","last_fullsized":"2023-06-23T18:35:54.526508+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:22.822277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526010+0000","last_change":"2023-06-23T18:35:53.131500+0000","last_active":"2023-06-23T18:35:54.526010+0000","last_peered":"2023-06-23T18:35:54.526010+0000","last_clean":"2023-06-23T18:35:54.526010+0000","last_became_active":"2023-06-23T18:35:53.131222+0000","last_became_peered":"2023-06-23T18:35:53.131222+0000","last_unstale":"2023-06-23T18:35:54.526010+0000","last_undegraded":"2023-06-23T18:35:54.526010+0000","last_fullsized":"2023-06-23T18:35:54.526010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:38.016680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527523+0000","last_change":"2023-06-23T18:35:53.532882+0000","last_active":"2023-06-23T18:35:54.527523+0000","last_peered":"2023-06-23T18:35:54.527523+0000","last_clean":"2023-06-23T18:35:54.527523+0000","last_became_active":"2023-06-23T18:35:53.532615+0000","last_became_peered":"2023-06-23T18:35:53.532615+0000","last_unstale":"2023-06-23T18:35:54.527523+0000","last_undegraded":"2023-06-23T18:35:54.527523+0000","last_fullsized":"2023-06-23T18:35:54.527523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257469+0000","last_change":"2023-06-23T18:35:48.735631+0000","last_active":"2023-06-23T18:35:55.257469+0000","last_peered":"2023-06-23T18:35:55.257469+0000","last_clean":"2023-06-23T18:35:55.257469+0000","last_became_active":"2023-06-23T18:35:48.735255+0000","last_became_peered":"2023-06-23T18:35:48.735255+0000","last_unstale":"2023-06-23T18:35:55.257469+0000","last_undegraded":"2023-06-23T18:35:55.257469+0000","last_fullsized":"2023-06-23T18:35:55.257469+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:32:56.020041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527397+0000","last_change":"2023-06-23T18:35:53.132237+0000","last_active":"2023-06-23T18:35:54.527397+0000","last_peered":"2023-06-23T18:35:54.527397+0000","last_clean":"2023-06-23T18:35:54.527397+0000","last_became_active":"2023-06-23T18:35:53.131968+0000","last_became_peered":"2023-06-23T18:35:53.131968+0000","last_unstale":"2023-06-23T18:35:54.527397+0000","last_undegraded":"2023-06-23T18:35:54.527397+0000","last_fullsized":"2023-06-23T18:35:54.527397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:27.976038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287780+0000","last_change":"2023-06-23T18:35:53.536675+0000","last_active":"2023-06-23T18:35:55.287780+0000","last_peered":"2023-06-23T18:35:55.287780+0000","last_clean":"2023-06-23T18:35:55.287780+0000","last_became_active":"2023-06-23T18:35:53.535843+0000","last_became_peered":"2023-06-23T18:35:53.535843+0000","last_unstale":"2023-06-23T18:35:55.287780+0000","last_undegraded":"2023-06-23T18:35:55.287780+0000","last_fullsized":"2023-06-23T18:35:55.287780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:07.647324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257359+0000","last_change":"2023-06-23T18:35:47.970359+0000","last_active":"2023-06-23T18:35:55.257359+0000","last_peered":"2023-06-23T18:35:55.257359+0000","last_clean":"2023-06-23T18:35:55.257359+0000","last_became_active":"2023-06-23T18:35:47.970118+0000","last_became_peered":"2023-06-23T18:35:47.970118+0000","last_unstale":"2023-06-23T18:35:55.257359+0000","last_undegraded":"2023-06-23T18:35:55.257359+0000","last_fullsized":"2023-06-23T18:35:55.257359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:35.810772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527072+0000","last_change":"2023-06-23T18:35:53.132066+0000","last_active":"2023-06-23T18:35:54.527072+0000","last_peered":"2023-06-23T18:35:54.527072+0000","last_clean":"2023-06-23T18:35:54.527072+0000","last_became_active":"2023-06-23T18:35:53.131788+0000","last_became_peered":"2023-06-23T18:35:53.131788+0000","last_unstale":"2023-06-23T18:35:54.527072+0000","last_undegraded":"2023-06-23T18:35:54.527072+0000","last_fullsized":"2023-06-23T18:35:54.527072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:30.042205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527690+0000","last_change":"2023-06-23T18:35:53.536630+0000","last_active":"2023-06-23T18:35:54.527690+0000","last_peered":"2023-06-23T18:35:54.527690+0000","last_clean":"2023-06-23T18:35:54.527690+0000","last_became_active":"2023-06-23T18:35:53.536298+0000","last_became_peered":"2023-06-23T18:35:53.536298+0000","last_unstale":"2023-06-23T18:35:54.527690+0000","last_undegraded":"2023-06-23T18:35:54.527690+0000","last_fullsized":"2023-06-23T18:35:54.527690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:16.869108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287850+0000","last_change":"2023-06-23T18:35:53.126505+0000","last_active":"2023-06-23T18:35:55.287850+0000","last_peered":"2023-06-23T18:35:55.287850+0000","last_clean":"2023-06-23T18:35:55.287850+0000","last_became_active":"2023-06-23T18:35:53.126226+0000","last_became_peered":"2023-06-23T18:35:53.126226+0000","last_unstale":"2023-06-23T18:35:55.287850+0000","last_undegraded":"2023-06-23T18:35:55.287850+0000","last_fullsized":"2023-06-23T18:35:55.287850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:40.964533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287939+0000","last_change":"2023-06-23T18:35:53.533402+0000","last_active":"2023-06-23T18:35:55.287939+0000","last_peered":"2023-06-23T18:35:55.287939+0000","last_clean":"2023-06-23T18:35:55.287939+0000","last_became_active":"2023-06-23T18:35:53.533163+0000","last_became_peered":"2023-06-23T18:35:53.533163+0000","last_unstale":"2023-06-23T18:35:55.287939+0000","last_undegraded":"2023-06-23T18:35:55.287939+0000","last_fullsized":"2023-06-23T18:35:55.287939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:01:29.821609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526906+0000","last_change":"2023-06-23T18:35:53.123684+0000","last_active":"2023-06-23T18:35:54.526906+0000","last_peered":"2023-06-23T18:35:54.526906+0000","last_clean":"2023-06-23T18:35:54.526906+0000","last_became_active":"2023-06-23T18:35:53.123369+0000","last_became_peered":"2023-06-23T18:35:53.123369+0000","last_unstale":"2023-06-23T18:35:54.526906+0000","last_undegraded":"2023-06-23T18:35:54.526906+0000","last_fullsized":"2023-06-23T18:35:54.526906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:18.604812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525869+0000","last_change":"2023-06-23T18:35:53.534162+0000","last_active":"2023-06-23T18:35:54.525869+0000","last_peered":"2023-06-23T18:35:54.525869+0000","last_clean":"2023-06-23T18:35:54.525869+0000","last_became_active":"2023-06-23T18:35:53.533853+0000","last_became_peered":"2023-06-23T18:35:53.533853+0000","last_unstale":"2023-06-23T18:35:54.525869+0000","last_undegraded":"2023-06-23T18:35:54.525869+0000","last_fullsized":"2023-06-23T18:35:54.525869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:30.583311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527209+0000","last_change":"2023-06-23T18:35:53.123532+0000","last_active":"2023-06-23T18:35:54.527209+0000","last_peered":"2023-06-23T18:35:54.527209+0000","last_clean":"2023-06-23T18:35:54.527209+0000","last_became_active":"2023-06-23T18:35:53.123228+0000","last_became_peered":"2023-06-23T18:35:53.123228+0000","last_unstale":"2023-06-23T18:35:54.527209+0000","last_undegraded":"2023-06-23T18:35:54.527209+0000","last_fullsized":"2023-06-23T18:35:54.527209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:29.581544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527107+0000","last_change":"2023-06-23T18:35:53.533902+0000","last_active":"2023-06-23T18:35:54.527107+0000","last_peered":"2023-06-23T18:35:54.527107+0000","last_clean":"2023-06-23T18:35:54.527107+0000","last_became_active":"2023-06-23T18:35:53.533563+0000","last_became_peered":"2023-06-23T18:35:53.533563+0000","last_unstale":"2023-06-23T18:35:54.527107+0000","last_undegraded":"2023-06-23T18:35:54.527107+0000","last_fullsized":"2023-06-23T18:35:54.527107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:55:12.340901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288866+0000","last_change":"2023-06-23T18:35:53.126195+0000","last_active":"2023-06-23T18:35:55.288866+0000","last_peered":"2023-06-23T18:35:55.288866+0000","last_clean":"2023-06-23T18:35:55.288866+0000","last_became_active":"2023-06-23T18:35:53.125886+0000","last_became_peered":"2023-06-23T18:35:53.125886+0000","last_unstale":"2023-06-23T18:35:55.288866+0000","last_undegraded":"2023-06-23T18:35:55.288866+0000","last_fullsized":"2023-06-23T18:35:55.288866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:57.730157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288616+0000","last_change":"2023-06-23T18:35:53.534747+0000","last_active":"2023-06-23T18:35:55.288616+0000","last_peered":"2023-06-23T18:35:55.288616+0000","last_clean":"2023-06-23T18:35:55.288616+0000","last_became_active":"2023-06-23T18:35:53.534495+0000","last_became_peered":"2023-06-23T18:35:53.534495+0000","last_unstale":"2023-06-23T18:35:55.288616+0000","last_undegraded":"2023-06-23T18:35:55.288616+0000","last_fullsized":"2023-06-23T18:35:55.288616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:30.583311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288416+0000","last_change":"2023-06-23T18:35:53.130647+0000","last_active":"2023-06-23T18:35:55.288416+0000","last_peered":"2023-06-23T18:35:55.288416+0000","last_clean":"2023-06-23T18:35:55.288416+0000","last_became_active":"2023-06-23T18:35:53.129738+0000","last_became_peered":"2023-06-23T18:35:53.129738+0000","last_unstale":"2023-06-23T18:35:55.288416+0000","last_undegraded":"2023-06-23T18:35:55.288416+0000","last_fullsized":"2023-06-23T18:35:55.288416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:16.518353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525676+0000","last_change":"2023-06-23T18:35:53.540648+0000","last_active":"2023-06-23T18:35:54.525676+0000","last_peered":"2023-06-23T18:35:54.525676+0000","last_clean":"2023-06-23T18:35:54.525676+0000","last_became_active":"2023-06-23T18:35:53.540380+0000","last_became_peered":"2023-06-23T18:35:53.540380+0000","last_unstale":"2023-06-23T18:35:54.525676+0000","last_undegraded":"2023-06-23T18:35:54.525676+0000","last_fullsized":"2023-06-23T18:35:54.525676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257206+0000","last_change":"2023-06-23T18:35:53.539029+0000","last_active":"2023-06-23T18:35:55.257206+0000","last_peered":"2023-06-23T18:35:55.257206+0000","last_clean":"2023-06-23T18:35:55.257206+0000","last_became_active":"2023-06-23T18:35:53.538640+0000","last_became_peered":"2023-06-23T18:35:53.538640+0000","last_unstale":"2023-06-23T18:35:55.257206+0000","last_undegraded":"2023-06-23T18:35:55.257206+0000","last_fullsized":"2023-06-23T18:35:55.257206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:52.658179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288459+0000","last_change":"2023-06-23T18:35:53.541022+0000","last_active":"2023-06-23T18:35:55.288459+0000","last_peered":"2023-06-23T18:35:55.288459+0000","last_clean":"2023-06-23T18:35:55.288459+0000","last_became_active":"2023-06-23T18:35:53.540744+0000","last_became_peered":"2023-06-23T18:35:53.540744+0000","last_unstale":"2023-06-23T18:35:55.288459+0000","last_undegraded":"2023-06-23T18:35:55.288459+0000","last_fullsized":"2023-06-23T18:35:55.288459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:31.043972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526923+0000","last_change":"2023-06-23T18:35:53.540406+0000","last_active":"2023-06-23T18:35:54.526923+0000","last_peered":"2023-06-23T18:35:54.526923+0000","last_clean":"2023-06-23T18:35:54.526923+0000","last_became_active":"2023-06-23T18:35:53.540125+0000","last_became_peered":"2023-06-23T18:35:53.540125+0000","last_unstale":"2023-06-23T18:35:54.526923+0000","last_undegraded":"2023-06-23T18:35:54.526923+0000","last_fullsized":"2023-06-23T18:35:54.526923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:45:17.669839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288781+0000","last_change":"2023-06-23T18:35:53.542899+0000","last_active":"2023-06-23T18:35:55.288781+0000","last_peered":"2023-06-23T18:35:55.288781+0000","last_clean":"2023-06-23T18:35:55.288781+0000","last_became_active":"2023-06-23T18:35:53.541798+0000","last_became_peered":"2023-06-23T18:35:53.541798+0000","last_unstale":"2023-06-23T18:35:55.288781+0000","last_undegraded":"2023-06-23T18:35:55.288781+0000","last_fullsized":"2023-06-23T18:35:55.288781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:30.349972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527379+0000","last_change":"2023-06-23T18:35:53.532923+0000","last_active":"2023-06-23T18:35:54.527379+0000","last_peered":"2023-06-23T18:35:54.527379+0000","last_clean":"2023-06-23T18:35:54.527379+0000","last_became_active":"2023-06-23T18:35:53.532674+0000","last_became_peered":"2023-06-23T18:35:53.532674+0000","last_unstale":"2023-06-23T18:35:54.527379+0000","last_undegraded":"2023-06-23T18:35:54.527379+0000","last_fullsized":"2023-06-23T18:35:54.527379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:17.016870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":823296,"data_stored":665215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":74428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":798720,"data_stored":664796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":73649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":200704,"data_stored":73067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":196608,"data_stored":67865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":76048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":75080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:35:57.991 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-06-23T18:35:57.992 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-06-23T18:35:57.992 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-06-23T18:35:57.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-06-23T18:35:57.993 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-06-23T18:35:58.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.137+0000 7f9b7877e700 1 Processor -- start 2023-06-23T18:35:58.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.137+0000 7f9b7877e700 1 -- start start 2023-06-23T18:35:58.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.138+0000 7f9b7877e700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9b70071490 0x7f9b70071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.138+0000 7f9b7877e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b701608f0 con 0x7f9b700aee30 2023-06-23T18:35:58.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.138+0000 7f9b7877e700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b70160e20 con 0x7f9b700b1270 2023-06-23T18:35:58.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.138+0000 7f9b7877e700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b7006acb0 con 0x7f9b70071490 2023-06-23T18:35:58.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.138+0000 7f9b76d1b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9b700aee30 0x7f9b7006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42218/0 (socket says 172.21.15.79:42218) 2023-06-23T18:35:58.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.138+0000 7f9b76d1b700 1 -- 172.21.15.79:0/3333229109 learned_addr learned my addr 172.21.15.79:0/3333229109 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:58.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 -- 172.21.15.79:0/3333229109 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181044146 0 0) 0x7f9b701608f0 con 0x7f9b700aee30 2023-06-23T18:35:58.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b7651a700 1 --2- 172.21.15.79:0/3333229109 >> v2:172.21.15.155:3300/0 conn(0x7f9b70071490 0x7f9b70071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 -- 172.21.15.79:0/3333229109 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b5c0035b0 con 0x7f9b700aee30 2023-06-23T18:35:58.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 -- 172.21.15.79:0/3333229109 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2795614907 0 0) 0x7f9b5c0035b0 con 0x7f9b700aee30 2023-06-23T18:35:58.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 -- 172.21.15.79:0/3333229109 >> v2:172.21.15.155:3300/0 conn(0x7f9b70071490 msgr2=0x7f9b70071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:58.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 --2- 172.21.15.79:0/3333229109 >> v2:172.21.15.155:3300/0 conn(0x7f9b70071490 0x7f9b70071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 -- 172.21.15.79:0/3333229109 >> v1:172.21.15.155:6789/0 conn(0x7f9b700b1270 legacy=0x7f9b70069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 -- 172.21.15.79:0/3333229109 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b700cd4f0 con 0x7f9b700aee30 2023-06-23T18:35:58.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 -- 172.21.15.79:0/3333229109 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f9b680019f0 con 0x7f9b700aee30 2023-06-23T18:35:58.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 -- 172.21.15.79:0/3333229109 >> v1:172.21.15.79:6789/0 conn(0x7f9b700aee30 legacy=0x7f9b7006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 --2- 172.21.15.79:0/3333229109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b5c003a00 0x7f9b5c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 --2- 172.21.15.79:0/3333229109 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b5c004850 0x7f9b5c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 --2- 172.21.15.79:0/3333229109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c005200 0x7f9b5c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 -- 172.21.15.79:0/3333229109 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7006acb0 con 0x7f9b5c003a00 2023-06-23T18:35:58.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 -- 172.21.15.79:0/3333229109 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c00c110 con 0x7f9b5c005200 2023-06-23T18:35:58.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75518700 1 -- 172.21.15.79:0/3333229109 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c00c2d0 con 0x7f9b5c004850 2023-06-23T18:35:58.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b7651a700 1 --2- 172.21.15.79:0/3333229109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b5c003a00 0x7f9b5c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b76d1b700 1 --2- 172.21.15.79:0/3333229109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c005200 0x7f9b5c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b75d19700 1 --2- 172.21.15.79:0/3333229109 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b5c004850 0x7f9b5c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b7651a700 1 -- 172.21.15.79:0/3333229109 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b5c004850 msgr2=0x7f9b5c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:58.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b7651a700 1 --2- 172.21.15.79:0/3333229109 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b5c004850 0x7f9b5c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b7651a700 1 -- 172.21.15.79:0/3333229109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c005200 msgr2=0x7f9b5c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:58.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b7651a700 1 --2- 172.21.15.79:0/3333229109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c005200 0x7f9b5c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b76d1b700 1 -- 172.21.15.79:0/3333229109 reap_dead start 2023-06-23T18:35:58.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b7651a700 1 -- 172.21.15.79:0/3333229109 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c0035b0 con 0x7f9b5c003a00 2023-06-23T18:35:58.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.139+0000 7f9b7651a700 1 --2- 172.21.15.79:0/3333229109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b5c003a00 0x7f9b5c003e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009b60 tx=0x7f9b5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0de0a151f5b61dd server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:58.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.140+0000 7f9b75518700 1 -- 172.21.15.79:0/3333229109 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6000e6c0 con 0x7f9b5c003a00 2023-06-23T18:35:58.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.140+0000 7f9b75518700 1 -- 172.21.15.79:0/3333229109 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6000e820 con 0x7f9b5c003a00 2023-06-23T18:35:58.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.140+0000 7f9b75518700 1 -- 172.21.15.79:0/3333229109 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6000eb90 con 0x7f9b5c003a00 2023-06-23T18:35:58.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.140+0000 7f9b7877e700 1 -- 172.21.15.79:0/3333229109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b5c003a00 msgr2=0x7f9b5c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.140+0000 7f9b7877e700 1 --2- 172.21.15.79:0/3333229109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b5c003a00 0x7f9b5c003e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009b60 tx=0x7f9b5c005880 comp rx=0 tx=0).stop 2023-06-23T18:35:58.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.140+0000 7f9b7877e700 1 -- 172.21.15.79:0/3333229109 shutdown_connections 2023-06-23T18:35:58.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.140+0000 7f9b7877e700 1 --2- 172.21.15.79:0/3333229109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b5c003a00 0x7f9b5c003e10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.140+0000 7f9b7877e700 1 -- 172.21.15.79:0/3333229109 >> 172.21.15.79:0/3333229109 conn(0x7f9b7006d660 msgr2=0x7f9b7006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:58.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.140+0000 7f9b7877e700 1 -- 172.21.15.79:0/3333229109 shutdown_connections 2023-06-23T18:35:58.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.140+0000 7f9b7877e700 1 -- 172.21.15.79:0/3333229109 wait complete. 2023-06-23T18:35:58.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.141+0000 7f9b7877e700 1 Processor -- start 2023-06-23T18:35:58.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.141+0000 7f9b7877e700 1 -- start start 2023-06-23T18:35:58.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.142+0000 7f9b7877e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c003a00 0x7f9b700b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.142+0000 7f9b7877e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b700b5360 0x7f9b70150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.142+0000 7f9b7877e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b700c6e90 0x7f9b700c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.142+0000 7f9b7877e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700c9880 con 0x7f9b700b5360 2023-06-23T18:35:58.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.142+0000 7f9b7877e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b701511d0 con 0x7f9b5c003a00 2023-06-23T18:35:58.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.142+0000 7f9b7877e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700b42c0 con 0x7f9b700c6e90 2023-06-23T18:35:58.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.142+0000 7f9b75d19700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b700b5360 0x7f9b70150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.142+0000 7f9b75d19700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b700b5360 0x7f9b70150c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39186/0 (socket says 172.21.15.79:39186) 2023-06-23T18:35:58.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.142+0000 7f9b75d19700 1 -- 172.21.15.79:0/595570277 learned_addr learned my addr 172.21.15.79:0/595570277 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:58.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.142+0000 7f9b7651a700 1 --2- 172.21.15.79:0/595570277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c003a00 0x7f9b700b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.142+0000 7f9b76d1b700 1 --2- 172.21.15.79:0/595570277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b700c6e90 0x7f9b700c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.143+0000 7f9b75d19700 1 -- 172.21.15.79:0/595570277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b700c6e90 msgr2=0x7f9b700c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.143+0000 7f9b75d19700 1 --2- 172.21.15.79:0/595570277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b700c6e90 0x7f9b700c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.143+0000 7f9b75d19700 1 -- 172.21.15.79:0/595570277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c003a00 msgr2=0x7f9b700b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.143+0000 7f9b75d19700 1 --2- 172.21.15.79:0/595570277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c003a00 0x7f9b700b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.143+0000 7f9b75d19700 1 -- 172.21.15.79:0/595570277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b700b45a0 con 0x7f9b700b5360 2023-06-23T18:35:58.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.143+0000 7f9b75d19700 1 --2- 172.21.15.79:0/595570277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b700b5360 0x7f9b70150c90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00e4e0 tx=0x7f9b6c00e7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:58.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.143+0000 7f9b66ffd700 1 -- 172.21.15.79:0/595570277 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c00ee70 con 0x7f9b700b5360 2023-06-23T18:35:58.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.143+0000 7f9b66ffd700 1 -- 172.21.15.79:0/595570277 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c0052e0 con 0x7f9b700b5360 2023-06-23T18:35:58.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.143+0000 7f9b7877e700 1 -- 172.21.15.79:0/595570277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b700b4880 con 0x7f9b700b5360 2023-06-23T18:35:58.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.143+0000 7f9b66ffd700 1 -- 172.21.15.79:0/595570277 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c003680 con 0x7f9b700b5360 2023-06-23T18:35:58.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.143+0000 7f9b7877e700 1 -- 172.21.15.79:0/595570277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b700b4e40 con 0x7f9b700b5360 2023-06-23T18:35:58.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.144+0000 7f9b7877e700 1 -- 172.21.15.79:0/595570277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b700b4fd0 con 0x7f9b700b5360 2023-06-23T18:35:58.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.144+0000 7f9b66ffd700 1 -- 172.21.15.79:0/595570277 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9b6c003800 con 0x7f9b700b5360 2023-06-23T18:35:58.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.145+0000 7f9b66ffd700 1 --2- 172.21.15.79:0/595570277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9b5406ace0 0x7f9b5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.145+0000 7f9b66ffd700 1 -- 172.21.15.79:0/595570277 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9b6c0889a0 con 0x7f9b700b5360 2023-06-23T18:35:58.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.145+0000 7f9b7651a700 1 --2- 172.21.15.79:0/595570277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9b5406ace0 0x7f9b5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.145+0000 7f9b7651a700 1 --2- 172.21.15.79:0/595570277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9b5406ace0 0x7f9b5406d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9b600090b0 tx=0x7f9b6001e040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:58.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.149+0000 7f9b66ffd700 1 -- 172.21.15.79:0/595570277 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9b6c053b00 con 0x7f9b700b5360 2023-06-23T18:35:58.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.397+0000 7f9b7877e700 1 -- 172.21.15.79:0/595570277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f9b700724a0 con 0x7f9b700b5360 2023-06-23T18:35:58.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.400+0000 7f9b66ffd700 1 -- 172.21.15.79:0/595570277 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170839 (secure 0 0 0) 0x7f9b6c019090 con 0x7f9b700b5360 2023-06-23T18:35:58.407 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:58.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.402+0000 7f9b7877e700 1 -- 172.21.15.79:0/595570277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9b5406ace0 msgr2=0x7f9b5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.402+0000 7f9b7877e700 1 --2- 172.21.15.79:0/595570277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9b5406ace0 0x7f9b5406d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9b600090b0 tx=0x7f9b6001e040 comp rx=0 tx=0).stop 2023-06-23T18:35:58.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.402+0000 7f9b7877e700 1 -- 172.21.15.79:0/595570277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b700b5360 msgr2=0x7f9b70150c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.402+0000 7f9b7877e700 1 --2- 172.21.15.79:0/595570277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b700b5360 0x7f9b70150c90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00e4e0 tx=0x7f9b6c00e7f0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.402+0000 7f9b7877e700 1 -- 172.21.15.79:0/595570277 shutdown_connections 2023-06-23T18:35:58.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.402+0000 7f9b7877e700 1 --2- 172.21.15.79:0/595570277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b700c6e90 0x7f9b700c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.402+0000 7f9b7877e700 1 --2- 172.21.15.79:0/595570277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9b5406ace0 0x7f9b5406d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.402+0000 7f9b7877e700 1 --2- 172.21.15.79:0/595570277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c003a00 0x7f9b700b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.402+0000 7f9b7877e700 1 --2- 172.21.15.79:0/595570277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b700b5360 0x7f9b70150c90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.402+0000 7f9b7877e700 1 -- 172.21.15.79:0/595570277 >> 172.21.15.79:0/595570277 conn(0x7f9b7006d660 msgr2=0x7f9b700cb850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:58.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.403+0000 7f9b7877e700 1 -- 172.21.15.79:0/595570277 shutdown_connections 2023-06-23T18:35:58.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.403+0000 7f9b7877e700 1 -- 172.21.15.79:0/595570277 wait complete. 2023-06-23T18:35:58.419 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":5,"active_gid":4103,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1804789972},{"type":"v1","addr":"172.21.15.79:6833","nonce":1804789972}]},"active_addr":"172.21.15.79:6833/1804789972","active_change":"2023-06-23T18:35:40.688825+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":1695416561}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":3377186696}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":96887912}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":668123001}]}]} 2023-06-23T18:35:58.424 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-06-23T18:35:58.425 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-06-23T18:35:58.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:35:58.545 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.540+0000 7fbefab21700 1 Processor -- start 2023-06-23T18:35:58.545 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.540+0000 7fbefab21700 1 -- start start 2023-06-23T18:35:58.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.540+0000 7fbefab21700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef415c110 0x7fbef41608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbefab21700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef4069e90 con 0x7fbef415c4e0 2023-06-23T18:35:58.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbefab21700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef406db00 con 0x7fbef4071220 2023-06-23T18:35:58.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbefab21700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40ccb30 con 0x7fbef415c110 2023-06-23T18:35:58.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef3fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fbef415c4e0 0x7fbef4070b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42236/0 (socket says 172.21.15.79:42236) 2023-06-23T18:35:58.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef3fff700 1 -- 172.21.15.79:0/4238998695 learned_addr learned my addr 172.21.15.79:0/4238998695 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:58.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef88bd700 1 --2- 172.21.15.79:0/4238998695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef415c110 0x7fbef41608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598116363 0 0) 0x7fbef4069e90 con 0x7fbef415c4e0 2023-06-23T18:35:58.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee40035b0 con 0x7fbef415c4e0 2023-06-23T18:35:58.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010275384 0 0) 0x7fbef406db00 con 0x7fbef4071220 2023-06-23T18:35:58.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbef4069e90 con 0x7fbef4071220 2023-06-23T18:35:58.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 670364269 0 0) 0x7fbee40035b0 con 0x7fbef415c4e0 2023-06-23T18:35:58.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef415c110 msgr2=0x7fbef41608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:58.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 --2- 172.21.15.79:0/4238998695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef415c110 0x7fbef41608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 >> v1:172.21.15.155:6789/0 conn(0x7fbef4071220 legacy=0x7fbef406a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40ccd40 con 0x7fbef415c4e0 2023-06-23T18:35:58.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fbee0001670 con 0x7fbef415c4e0 2023-06-23T18:35:58.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 >> v1:172.21.15.79:6789/0 conn(0x7fbef415c4e0 legacy=0x7fbef4070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 --2- 172.21.15.79:0/4238998695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbee4003bc0 0x7fbee4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 --2- 172.21.15.79:0/4238998695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbee4004850 0x7fbee4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 --2- 172.21.15.79:0/4238998695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef415c110 0x7fbee4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40ccb30 con 0x7fbee4004850 2023-06-23T18:35:58.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee400bac0 con 0x7fbee4003bc0 2023-06-23T18:35:58.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee400bc30 con 0x7fbef415c110 2023-06-23T18:35:58.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef3fff700 1 --2- 172.21.15.79:0/4238998695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbee4004850 0x7fbee4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef88bd700 1 --2- 172.21.15.79:0/4238998695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbee4003bc0 0x7fbee4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.541+0000 7fbef90be700 1 --2- 172.21.15.79:0/4238998695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef415c110 0x7fbee4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbef3fff700 1 -- 172.21.15.79:0/4238998695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef415c110 msgr2=0x7fbee4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:58.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbef3fff700 1 --2- 172.21.15.79:0/4238998695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef415c110 0x7fbee4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbef3fff700 1 -- 172.21.15.79:0/4238998695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbee4003bc0 msgr2=0x7fbee4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:58.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbef3fff700 1 --2- 172.21.15.79:0/4238998695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbee4003bc0 0x7fbee4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbef3fff700 1 -- 172.21.15.79:0/4238998695 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee40035b0 con 0x7fbee4004850 2023-06-23T18:35:58.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbef3fff700 1 --2- 172.21.15.79:0/4238998695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbee4004850 0x7fbee4004cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbee000c240 tx=0x7fbee4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8ce49eedb8da8b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:58.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee000cd80 con 0x7fbee4004850 2023-06-23T18:35:58.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee0005960 con 0x7fbee4004850 2023-06-23T18:35:58.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbef37fe700 1 -- 172.21.15.79:0/4238998695 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0005d10 con 0x7fbee4004850 2023-06-23T18:35:58.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbefab21700 1 -- 172.21.15.79:0/4238998695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbee4004850 msgr2=0x7fbee4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbefab21700 1 --2- 172.21.15.79:0/4238998695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbee4004850 0x7fbee4004cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbee000c240 tx=0x7fbee4005880 comp rx=0 tx=0).stop 2023-06-23T18:35:58.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbef90be700 1 -- 172.21.15.79:0/4238998695 reap_dead start 2023-06-23T18:35:58.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbefab21700 1 -- 172.21.15.79:0/4238998695 shutdown_connections 2023-06-23T18:35:58.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbefab21700 1 -- 172.21.15.79:0/4238998695 >> 172.21.15.79:0/4238998695 conn(0x7fbef40b17e0 msgr2=0x7fbef406cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:58.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbefab21700 1 -- 172.21.15.79:0/4238998695 shutdown_connections 2023-06-23T18:35:58.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.542+0000 7fbefab21700 1 -- 172.21.15.79:0/4238998695 wait complete. 2023-06-23T18:35:58.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.544+0000 7fbefab21700 1 Processor -- start 2023-06-23T18:35:58.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.544+0000 7fbefab21700 1 -- start start 2023-06-23T18:35:58.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.544+0000 7fbefab21700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbee4004850 0x7fbef414e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.544+0000 7fbefab21700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbef40c34a0 0x7fbef40c3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.544+0000 7fbefab21700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef40c4c40 0x7fbef40c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.544+0000 7fbefab21700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40c55f0 con 0x7fbef40c34a0 2023-06-23T18:35:58.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.544+0000 7fbefab21700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40c6380 con 0x7fbef40c4c40 2023-06-23T18:35:58.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.544+0000 7fbefab21700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40c6030 con 0x7fbee4004850 2023-06-23T18:35:58.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbef40c34a0 0x7fbef40c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbef40c34a0 0x7fbef40c3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39206/0 (socket says 172.21.15.79:39206) 2023-06-23T18:35:58.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef3fff700 1 -- 172.21.15.79:0/3120260294 learned_addr learned my addr 172.21.15.79:0/3120260294 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:58.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef88bd700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbee4004850 0x7fbef414e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef90be700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef40c4c40 0x7fbef40c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef3fff700 1 -- 172.21.15.79:0/3120260294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbee4004850 msgr2=0x7fbef414e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef3fff700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbee4004850 0x7fbef414e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef3fff700 1 -- 172.21.15.79:0/3120260294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef40c4c40 msgr2=0x7fbef40c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef3fff700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef40c4c40 0x7fbef40c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef3fff700 1 -- 172.21.15.79:0/3120260294 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40c3f40 con 0x7fbef40c34a0 2023-06-23T18:35:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef88bd700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbee4004850 0x7fbef414e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:35:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef90be700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef40c4c40 0x7fbef40c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:35:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef3fff700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbef40c34a0 0x7fbef40c3910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbee4004780 tx=0x7fbef40cd300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef17fa700 1 -- 172.21.15.79:0/3120260294 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0004870 con 0x7fbef40c34a0 2023-06-23T18:35:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef17fa700 1 -- 172.21.15.79:0/3120260294 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee00049f0 con 0x7fbef40c34a0 2023-06-23T18:35:58.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbefab21700 1 -- 172.21.15.79:0/3120260294 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef40c4220 con 0x7fbef40c34a0 2023-06-23T18:35:58.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbef17fa700 1 -- 172.21.15.79:0/3120260294 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0004da0 con 0x7fbef40c34a0 2023-06-23T18:35:58.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.545+0000 7fbefab21700 1 -- 172.21.15.79:0/3120260294 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef414e8c0 con 0x7fbef40c34a0 2023-06-23T18:35:58.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.546+0000 7fbefab21700 1 -- 172.21.15.79:0/3120260294 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef40c6530 con 0x7fbef40c34a0 2023-06-23T18:35:58.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.547+0000 7fbef17fa700 1 -- 172.21.15.79:0/3120260294 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbee0007070 con 0x7fbef40c34a0 2023-06-23T18:35:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.547+0000 7fbef17fa700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbed806ace0 0x7fbed806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.547+0000 7fbef17fa700 1 -- 172.21.15.79:0/3120260294 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbee008ac10 con 0x7fbef40c34a0 2023-06-23T18:35:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.547+0000 7fbef88bd700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbed806ace0 0x7fbed806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.547+0000 7fbef88bd700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbed806ace0 0x7fbed806d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009b00 tx=0x7fbee8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.551+0000 7fbef17fa700 1 -- 172.21.15.79:0/3120260294 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbef40c6530 con 0x7fbef40c34a0 2023-06-23T18:35:58.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.737+0000 7fbefab21700 1 -- 172.21.15.79:0/3120260294 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbef40cd9c0 con 0x7fbef40c34a0 2023-06-23T18:35:58.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.738+0000 7fbef17fa700 1 -- 172.21.15.79:0/3120260294 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7fbee0055df0 con 0x7fbef40c34a0 2023-06-23T18:35:58.744 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:58.744 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:35:58.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.740+0000 7fbefab21700 1 -- 172.21.15.79:0/3120260294 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbed806ace0 msgr2=0x7fbed806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.740+0000 7fbefab21700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbed806ace0 0x7fbed806d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009b00 tx=0x7fbee8008040 comp rx=0 tx=0).stop 2023-06-23T18:35:58.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.740+0000 7fbefab21700 1 -- 172.21.15.79:0/3120260294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbef40c34a0 msgr2=0x7fbef40c3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.740+0000 7fbefab21700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbef40c34a0 0x7fbef40c3910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbee4004780 tx=0x7fbef40cd300 comp rx=0 tx=0).stop 2023-06-23T18:35:58.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.741+0000 7fbefab21700 1 -- 172.21.15.79:0/3120260294 shutdown_connections 2023-06-23T18:35:58.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.741+0000 7fbefab21700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbed806ace0 0x7fbed806d190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.741+0000 7fbefab21700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef40c4c40 0x7fbef40c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.741+0000 7fbefab21700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbee4004850 0x7fbef414e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.741+0000 7fbefab21700 1 --2- 172.21.15.79:0/3120260294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbef40c34a0 0x7fbef40c3910 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.741+0000 7fbefab21700 1 -- 172.21.15.79:0/3120260294 >> 172.21.15.79:0/3120260294 conn(0x7fbef40b17e0 msgr2=0x7fbef4070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:58.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.741+0000 7fbefab21700 1 -- 172.21.15.79:0/3120260294 shutdown_connections 2023-06-23T18:35:58.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.741+0000 7fbefab21700 1 -- 172.21.15.79:0/3120260294 wait complete. 2023-06-23T18:35:58.758 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-06-23T18:35:58.758 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:35:58.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:35:58.815 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:35:58.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.875+0000 7fa0a21d7700 1 Processor -- start 2023-06-23T18:35:58.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.875+0000 7fa0a21d7700 1 -- start start 2023-06-23T18:35:58.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.875+0000 7fa0a21d7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa09c15a380 0x7fa09c0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.875+0000 7fa0a21d7700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa09c0cea30 0x7fa09c0b1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa0a21d7700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09c0cee20 con 0x7fa09c15a750 2023-06-23T18:35:58.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa0a21d7700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09c0b1d40 con 0x7fa09c15a380 2023-06-23T18:35:58.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa0a21d7700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09c0b2020 con 0x7fa09c0cea30 2023-06-23T18:35:58.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa09b7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa09c15a380 0x7fa09c0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa09b7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa09c15a380 0x7fa09c0ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39212/0 (socket says 172.21.15.79:39212) 2023-06-23T18:35:58.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa09b7fe700 1 -- 172.21.15.79:0/4030093843 learned_addr learned my addr 172.21.15.79:0/4030093843 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:58.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa09bfff700 1 --2- 172.21.15.79:0/4030093843 >> v2:172.21.15.155:3300/0 conn(0x7fa09c0cea30 0x7fa09c0b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa09b7fe700 1 -- 172.21.15.79:0/4030093843 >> v2:172.21.15.155:3300/0 conn(0x7fa09c0cea30 msgr2=0x7fa09c0b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:58.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa09b7fe700 1 --2- 172.21.15.79:0/4030093843 >> v2:172.21.15.155:3300/0 conn(0x7fa09c0cea30 0x7fa09c0b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa09b7fe700 1 -- 172.21.15.79:0/4030093843 >> v1:172.21.15.155:6789/0 conn(0x7fa09c15a750 legacy=0x7fa09c0ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa09b7fe700 1 -- 172.21.15.79:0/4030093843 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c0ad650 con 0x7fa09c15a380 2023-06-23T18:35:58.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa09b7fe700 1 --2- 172.21.15.79:0/4030093843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa09c15a380 0x7fa09c0ce4f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa09c0c43f0 tx=0x7fa08c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af93e5cb6f5a925 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:58.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa09a7fc700 1 -- 172.21.15.79:0/4030093843 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c005e30 con 0x7fa09c15a380 2023-06-23T18:35:58.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa09a7fc700 1 -- 172.21.15.79:0/4030093843 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c004030 con 0x7fa09c15a380 2023-06-23T18:35:58.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.876+0000 7fa09a7fc700 1 -- 172.21.15.79:0/4030093843 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0043e0 con 0x7fa09c15a380 2023-06-23T18:35:58.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.877+0000 7fa0a21d7700 1 -- 172.21.15.79:0/4030093843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa09c15a380 msgr2=0x7fa09c0ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.877+0000 7fa0a21d7700 1 --2- 172.21.15.79:0/4030093843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa09c15a380 0x7fa09c0ce4f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa09c0c43f0 tx=0x7fa08c009bd0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.877+0000 7fa0a21d7700 1 -- 172.21.15.79:0/4030093843 shutdown_connections 2023-06-23T18:35:58.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.877+0000 7fa0a21d7700 1 --2- 172.21.15.79:0/4030093843 >> v2:172.21.15.155:3300/0 conn(0x7fa09c0cea30 0x7fa09c0b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.877+0000 7fa0a21d7700 1 --2- 172.21.15.79:0/4030093843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa09c15a380 0x7fa09c0ce4f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.877+0000 7fa0a21d7700 1 -- 172.21.15.79:0/4030093843 >> 172.21.15.79:0/4030093843 conn(0x7fa09c089fc0 msgr2=0x7fa09c099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:58.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.877+0000 7fa0a21d7700 1 -- 172.21.15.79:0/4030093843 shutdown_connections 2023-06-23T18:35:58.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.877+0000 7fa0a21d7700 1 -- 172.21.15.79:0/4030093843 wait complete. 2023-06-23T18:35:58.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.878+0000 7fa0a21d7700 1 Processor -- start 2023-06-23T18:35:58.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.878+0000 7fa0a21d7700 1 -- start start 2023-06-23T18:35:58.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa0a21d7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa09c0cea30 0x7fa09c0c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa0a21d7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa09c15a380 0x7fa09c155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa0a21d7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa09c15a750 0x7fa09c09a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa0a21d7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0b4840 con 0x7fa09c0cea30 2023-06-23T18:35:58.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa0a21d7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0b3bc0 con 0x7fa09c15a380 2023-06-23T18:35:58.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa0a21d7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0b3ea0 con 0x7fa09c15a750 2023-06-23T18:35:58.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa09b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa09c0cea30 0x7fa09c0c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa09b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa09c0cea30 0x7fa09c0c4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39218/0 (socket says 172.21.15.79:39218) 2023-06-23T18:35:58.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa09b7fe700 1 -- 172.21.15.79:0/820671231 learned_addr learned my addr 172.21.15.79:0/820671231 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:58.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa09b7fe700 1 -- 172.21.15.79:0/820671231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa09c15a750 msgr2=0x7fa09c09a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa09bfff700 1 --2- 172.21.15.79:0/820671231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa09c15a750 0x7fa09c09a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa09affd700 1 --2- 172.21.15.79:0/820671231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa09c15a380 0x7fa09c155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa09b7fe700 1 --2- 172.21.15.79:0/820671231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa09c15a750 0x7fa09c09a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa09b7fe700 1 -- 172.21.15.79:0/820671231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa09c15a380 msgr2=0x7fa09c155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:58.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa09b7fe700 1 --2- 172.21.15.79:0/820671231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa09c15a380 0x7fa09c155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:58.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.879+0000 7fa09b7fe700 1 -- 172.21.15.79:0/820671231 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c009500 con 0x7fa09c0cea30 2023-06-23T18:35:58.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.880+0000 7fa09b7fe700 1 --2- 172.21.15.79:0/820671231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa09c0cea30 0x7fa09c0c4df0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009c10 tx=0x7fa08c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:58.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.880+0000 7fa083fff700 1 -- 172.21.15.79:0/820671231 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c003d90 con 0x7fa09c0cea30 2023-06-23T18:35:58.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.880+0000 7fa083fff700 1 -- 172.21.15.79:0/820671231 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c01c420 con 0x7fa09c0cea30 2023-06-23T18:35:58.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.880+0000 7fa0a21d7700 1 -- 172.21.15.79:0/820671231 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c0b4180 con 0x7fa09c0cea30 2023-06-23T18:35:58.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.880+0000 7fa083fff700 1 -- 172.21.15.79:0/820671231 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c01c7b0 con 0x7fa09c0cea30 2023-06-23T18:35:58.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.880+0000 7fa0a21d7700 1 -- 172.21.15.79:0/820671231 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c0c5330 con 0x7fa09c0cea30 2023-06-23T18:35:58.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.881+0000 7fa0a21d7700 1 -- 172.21.15.79:0/820671231 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09c0b4980 con 0x7fa09c0cea30 2023-06-23T18:35:58.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.881+0000 7fa083fff700 1 -- 172.21.15.79:0/820671231 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa08c01c910 con 0x7fa09c0cea30 2023-06-23T18:35:58.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.882+0000 7fa083fff700 1 --2- 172.21.15.79:0/820671231 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa08806ace0 0x7fa08806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:58.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.882+0000 7fa083fff700 1 -- 172.21.15.79:0/820671231 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa08c08a4b0 con 0x7fa09c0cea30 2023-06-23T18:35:58.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.882+0000 7fa09affd700 1 --2- 172.21.15.79:0/820671231 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa08806ace0 0x7fa08806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:58.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.882+0000 7fa09affd700 1 --2- 172.21.15.79:0/820671231 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa08806ace0 0x7fa08806d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa09c0c3fe0 tx=0x7fa090009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:58.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:58.886+0000 7fa083fff700 1 -- 172.21.15.79:0/820671231 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa09c0b4980 con 0x7fa09c0cea30 2023-06-23T18:35:59.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.073+0000 7fa0a21d7700 1 -- 172.21.15.79:0/820671231 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa09c04e240 con 0x7fa09c0cea30 2023-06-23T18:35:59.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.074+0000 7fa083fff700 1 -- 172.21.15.79:0/820671231 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7fa08c055570 con 0x7fa09c0cea30 2023-06-23T18:35:59.080 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:35:59.080 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:35:59.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.076+0000 7fa0a21d7700 1 -- 172.21.15.79:0/820671231 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa08806ace0 msgr2=0x7fa08806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.076+0000 7fa0a21d7700 1 --2- 172.21.15.79:0/820671231 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa08806ace0 0x7fa08806d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa09c0c3fe0 tx=0x7fa090009040 comp rx=0 tx=0).stop 2023-06-23T18:35:59.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.076+0000 7fa0a21d7700 1 -- 172.21.15.79:0/820671231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa09c0cea30 msgr2=0x7fa09c0c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.076+0000 7fa0a21d7700 1 --2- 172.21.15.79:0/820671231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa09c0cea30 0x7fa09c0c4df0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009c10 tx=0x7fa08c004750 comp rx=0 tx=0).stop 2023-06-23T18:35:59.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.076+0000 7fa0a21d7700 1 -- 172.21.15.79:0/820671231 shutdown_connections 2023-06-23T18:35:59.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.076+0000 7fa0a21d7700 1 --2- 172.21.15.79:0/820671231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa09c15a750 0x7fa09c09a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.076+0000 7fa0a21d7700 1 --2- 172.21.15.79:0/820671231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa09c0cea30 0x7fa09c0c4df0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.076+0000 7fa0a21d7700 1 --2- 172.21.15.79:0/820671231 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa08806ace0 0x7fa08806d190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.076+0000 7fa0a21d7700 1 --2- 172.21.15.79:0/820671231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa09c15a380 0x7fa09c155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.076+0000 7fa0a21d7700 1 -- 172.21.15.79:0/820671231 >> 172.21.15.79:0/820671231 conn(0x7fa09c089fc0 msgr2=0x7fa09c0732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.076+0000 7fa0a21d7700 1 -- 172.21.15.79:0/820671231 shutdown_connections 2023-06-23T18:35:59.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.076+0000 7fa0a21d7700 1 -- 172.21.15.79:0/820671231 wait complete. 2023-06-23T18:35:59.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-06-23T18:35:59.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-06-23T18:35:59.095 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-06-23T18:35:59.096 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-06-23T18:35:59.096 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-06-23T18:35:59.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-06-23T18:35:59.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-06-23T18:35:59.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-06-23T18:35:59.110 INFO:tasks.ceph.mds.a.smithi079.stderr:2023-06-23T18:35:59.105+0000 7fd70b612700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-23T18:35:59.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.309+0000 7f929f89c700 1 Processor -- start 2023-06-23T18:35:59.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.310+0000 7f929f89c700 1 -- start start 2023-06-23T18:35:59.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.311+0000 7f929f89c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9298159170 0x7f92980cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.311+0000 7f929f89c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92980cd900 0x7f92980ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.311+0000 7f929f89c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f92980ce620 0x7f9298071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.311+0000 7f929f89c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92980cdda0 con 0x7f9298159170 2023-06-23T18:35:59.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.311+0000 7f929f89c700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f929815cd50 con 0x7f92980ce620 2023-06-23T18:35:59.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.311+0000 7f929f89c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929815cfa0 con 0x7f92980cd900 2023-06-23T18:35:59.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.311+0000 7f929ce37700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92980cd900 0x7f92980ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.311+0000 7f929de39700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f92980ce620 0x7f9298071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.311+0000 7f929ce37700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92980cd900 0x7f92980ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:60230/0 (socket says 172.21.15.79:60230) 2023-06-23T18:35:59.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.311+0000 7f929de39700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f92980ce620 0x7f9298071490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:45456/0 (socket says 172.21.15.79:45456) 2023-06-23T18:35:59.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.311+0000 7f929ce37700 1 -- 172.21.15.79:0/2210863446 learned_addr learned my addr 172.21.15.79:0/2210863446 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.312+0000 7f929ce37700 1 -- 172.21.15.79:0/2210863446 >> v2:172.21.15.155:3300/0 conn(0x7f92980ce620 msgr2=0x7f9298071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.312+0000 7f929ce37700 1 --2- 172.21.15.79:0/2210863446 >> v2:172.21.15.155:3300/0 conn(0x7f92980ce620 0x7f9298071490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.312+0000 7f929ce37700 1 -- 172.21.15.79:0/2210863446 >> v2:172.21.15.79:3300/0 conn(0x7f9298159170 msgr2=0x7f92980cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.312+0000 7f929ce37700 1 --2- 172.21.15.79:0/2210863446 >> v2:172.21.15.79:3300/0 conn(0x7f9298159170 0x7f92980cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.312+0000 7f929ce37700 1 -- 172.21.15.79:0/2210863446 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929815d280 con 0x7f92980cd900 2023-06-23T18:35:59.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.311+0000 7f548a531700 1 Processor -- start 2023-06-23T18:35:59.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.312+0000 7f548a531700 1 -- start start 2023-06-23T18:35:59.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.313+0000 7f548a531700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f54841608e0 0x7f5484069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.313+0000 7f548a531700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f54840c4dd0 0x7f548406a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.313+0000 7f548a531700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5484164890 con 0x7f54840c52f0 2023-06-23T18:35:59.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.313+0000 7f548a531700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54840c91c0 con 0x7f54841608e0 2023-06-23T18:35:59.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.313+0000 7f548a531700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5484164b70 con 0x7f54840c4dd0 2023-06-23T18:35:59.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.313+0000 7f5488ace700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f54840c4dd0 0x7f548406a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.313+0000 7f5488ace700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f54840c4dd0 0x7f548406a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:45462/0 (socket says 172.21.15.79:45462) 2023-06-23T18:35:59.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.313+0000 7f5488ace700 1 -- 172.21.15.79:0/3128058644 learned_addr learned my addr 172.21.15.79:0/3128058644 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.314+0000 7f929ce37700 1 --2- 172.21.15.79:0/2210863446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92980cd900 0x7f92980ce0e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f92980c3250 tx=0x7f9294009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47b5bdb3ba9d008a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.314+0000 7f5483fff700 1 --2- 172.21.15.79:0/3128058644 >> v2:172.21.15.79:3300/0 conn(0x7f54841608e0 0x7f5484069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.314+0000 7f5482ffd700 1 -- 172.21.15.79:0/3128058644 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782272194 0 0) 0x7f5484164890 con 0x7f54840c52f0 2023-06-23T18:35:59.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.314+0000 7f928ffff700 1 -- 172.21.15.79:0/2210863446 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929400b870 con 0x7f92980cd900 2023-06-23T18:35:59.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.314+0000 7f928ffff700 1 -- 172.21.15.79:0/2210863446 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929400b9d0 con 0x7f92980cd900 2023-06-23T18:35:59.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.314+0000 7f928ffff700 1 -- 172.21.15.79:0/2210863446 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929400bd60 con 0x7f92980cd900 2023-06-23T18:35:59.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.314+0000 7f3db597b700 1 Processor -- start 2023-06-23T18:35:59.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.315+0000 7f3db597b700 1 -- start start 2023-06-23T18:35:59.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.315+0000 7f929f89c700 1 -- 172.21.15.79:0/2210863446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92980cd900 msgr2=0x7f92980ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.315+0000 7f929f89c700 1 --2- 172.21.15.79:0/2210863446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92980cd900 0x7f92980ce0e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f92980c3250 tx=0x7f9294009bd0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.314+0000 7f5482ffd700 1 -- 172.21.15.79:0/3128058644 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f546c0035b0 con 0x7f54840c52f0 2023-06-23T18:35:59.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.314+0000 7f5488ace700 1 -- 172.21.15.79:0/3128058644 >> v2:172.21.15.79:3300/0 conn(0x7f54841608e0 msgr2=0x7f5484069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.314+0000 7f5488ace700 1 --2- 172.21.15.79:0/3128058644 >> v2:172.21.15.79:3300/0 conn(0x7f54841608e0 0x7f5484069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.314+0000 7f5488ace700 1 -- 172.21.15.79:0/3128058644 >> v1:172.21.15.79:6789/0 conn(0x7f54840c52f0 legacy=0x7f5484070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.314+0000 7f5488ace700 1 -- 172.21.15.79:0/3128058644 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5484164e50 con 0x7f54840c4dd0 2023-06-23T18:35:59.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.314+0000 7f5488ace700 1 --2- 172.21.15.79:0/3128058644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54840c4dd0 0x7f548406a3d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f54840712d0 tx=0x7f547c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea0de2d24318c8eb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.315+0000 7f5482ffd700 1 -- 172.21.15.79:0/3128058644 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c00e070 con 0x7f54840c4dd0 2023-06-23T18:35:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.315+0000 7f5482ffd700 1 -- 172.21.15.79:0/3128058644 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c00b910 con 0x7f54840c4dd0 2023-06-23T18:35:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.315+0000 7f5482ffd700 1 -- 172.21.15.79:0/3128058644 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c00bc80 con 0x7f54840c4dd0 2023-06-23T18:35:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.315+0000 7f929f89c700 1 -- 172.21.15.79:0/2210863446 shutdown_connections 2023-06-23T18:35:59.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.315+0000 7f929f89c700 1 --2- 172.21.15.79:0/2210863446 >> v2:172.21.15.155:3300/0 conn(0x7f92980ce620 0x7f9298071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.315+0000 7f929f89c700 1 --2- 172.21.15.79:0/2210863446 >> v2:172.21.15.79:3300/0 conn(0x7f9298159170 0x7f92980cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.315+0000 7f929f89c700 1 --2- 172.21.15.79:0/2210863446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92980cd900 0x7f92980ce0e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.315+0000 7f929f89c700 1 -- 172.21.15.79:0/2210863446 >> 172.21.15.79:0/2210863446 conn(0x7f929806d660 msgr2=0x7f929806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f929f89c700 1 -- 172.21.15.79:0/2210863446 shutdown_connections 2023-06-23T18:35:59.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f929f89c700 1 -- 172.21.15.79:0/2210863446 wait complete. 2023-06-23T18:35:59.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.315+0000 7f5cada2b700 1 Processor -- start 2023-06-23T18:35:59.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f5cada2b700 1 -- start start 2023-06-23T18:35:59.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f3db597b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3db015c4b0 0x7f3db00c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f3db597b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db00c91c0 con 0x7f3db00c3f40 2023-06-23T18:35:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f3db597b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db00cd350 con 0x7f3db00ca5e0 2023-06-23T18:35:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f3db597b700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db00c93f0 con 0x7f3db015c4b0 2023-06-23T18:35:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f3dae7fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3db00c3f40 0x7f3db00c9ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42280/0 (socket says 172.21.15.79:42280) 2023-06-23T18:35:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f3dae7fc700 1 -- 172.21.15.79:0/1636927561 learned_addr learned my addr 172.21.15.79:0/1636927561 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f3daeffd700 1 --2- 172.21.15.79:0/1636927561 >> v2:172.21.15.155:3300/0 conn(0x7f3db015c4b0 0x7f3db00c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f548a531700 1 -- 172.21.15.79:0/3128058644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54840c4dd0 msgr2=0x7f548406a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f548a531700 1 --2- 172.21.15.79:0/3128058644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54840c4dd0 0x7f548406a3d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f54840712d0 tx=0x7f547c00b330 comp rx=0 tx=0).stop 2023-06-23T18:35:59.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f3dadffb700 1 -- 172.21.15.79:0/1636927561 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289031337 0 0) 0x7f3db00c91c0 con 0x7f3db00c3f40 2023-06-23T18:35:59.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f3dadffb700 1 -- 172.21.15.79:0/1636927561 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d980035b0 con 0x7f3db00c3f40 2023-06-23T18:35:59.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f3dadffb700 1 -- 172.21.15.79:0/1636927561 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1264138218 0 0) 0x7f3db00cd350 con 0x7f3db00ca5e0 2023-06-23T18:35:59.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.316+0000 7f3dadffb700 1 -- 172.21.15.79:0/1636927561 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3db00c91c0 con 0x7f3db00ca5e0 2023-06-23T18:35:59.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7fd739103700 1 Processor -- start 2023-06-23T18:35:59.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f3daeffd700 1 -- 172.21.15.79:0/1636927561 >> v1:172.21.15.155:6789/0 conn(0x7f3db00ca5e0 legacy=0x7f3db0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f3daeffd700 1 -- 172.21.15.79:0/1636927561 >> v1:172.21.15.79:6789/0 conn(0x7f3db00c3f40 legacy=0x7f3db00c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f3daeffd700 1 -- 172.21.15.79:0/1636927561 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db00c96d0 con 0x7f3db015c4b0 2023-06-23T18:35:59.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f5cada2b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5ca81608e0 0x7f5ca8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f5cada2b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca80c4dd0 0x7f5ca806a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f5cada2b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca8164890 con 0x7f5ca80c52f0 2023-06-23T18:35:59.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f5cada2b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca80c91c0 con 0x7f5ca81608e0 2023-06-23T18:35:59.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f5cada2b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca8164b70 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f3daeffd700 1 --2- 172.21.15.79:0/1636927561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db015c4b0 0x7f3db00c3a00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009840 tx=0x7f3da4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c19588c97a41e3c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f5ca77fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca80c4dd0 0x7f5ca806a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f5ca77fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca80c4dd0 0x7f5ca806a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:60248/0 (socket says 172.21.15.79:60248) 2023-06-23T18:35:59.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f5ca77fe700 1 -- 172.21.15.79:0/3701033040 learned_addr learned my addr 172.21.15.79:0/3701033040 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f5ca77fe700 1 -- 172.21.15.79:0/3701033040 >> v2:172.21.15.79:3300/0 conn(0x7f5ca81608e0 msgr2=0x7f5ca8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f5ca77fe700 1 --2- 172.21.15.79:0/3701033040 >> v2:172.21.15.79:3300/0 conn(0x7f5ca81608e0 0x7f5ca8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f5ca77fe700 1 -- 172.21.15.79:0/3701033040 >> v1:172.21.15.155:6789/0 conn(0x7f5ca80c52f0 legacy=0x7f5ca8070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f5ca77fe700 1 -- 172.21.15.79:0/3701033040 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca8164e50 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f929f89c700 1 Processor -- start 2023-06-23T18:35:59.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7f929f89c700 1 -- start start 2023-06-23T18:35:59.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7f5ca77fe700 1 --2- 172.21.15.79:0/3701033040 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca80c4dd0 0x7f5ca806a3d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0009e50 tx=0x7f5ca000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a0acd2a2e1f4cc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.317+0000 7fd739103700 1 -- start start 2023-06-23T18:35:59.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7fd739103700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd7340adce0 0x7fd734069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7fd739103700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd73415ccc0 0x7fd73406a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7fd739103700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd734159a60 con 0x7fd734158a90 2023-06-23T18:35:59.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7fd739103700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd734159780 con 0x7fd7340adce0 2023-06-23T18:35:59.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7fd739103700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd73415c1e0 con 0x7fd73415ccc0 2023-06-23T18:35:59.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7f3dadffb700 1 -- 172.21.15.79:0/1636927561 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4004230 con 0x7f3db015c4b0 2023-06-23T18:35:59.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7f3dadffb700 1 -- 172.21.15.79:0/1636927561 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da4004390 con 0x7f3db015c4b0 2023-06-23T18:35:59.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7f3dadffb700 1 -- 172.21.15.79:0/1636927561 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da40036c0 con 0x7f3db015c4b0 2023-06-23T18:35:59.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f3db597b700 1 -- 172.21.15.79:0/1636927561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db015c4b0 msgr2=0x7f3db00c3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f3db597b700 1 --2- 172.21.15.79:0/1636927561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db015c4b0 0x7f3db00c3a00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009840 tx=0x7f3da4009b50 comp rx=0 tx=0).stop 2023-06-23T18:35:59.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7f929f89c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92980cd900 0x7f92980b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7f929f89c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92980ce620 0x7f9298144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7f929f89c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9298159170 0x7f929814e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7f929f89c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9298157670 con 0x7f92980ce620 2023-06-23T18:35:59.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7f929f89c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9298144d60 con 0x7f92980cd900 2023-06-23T18:35:59.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7f929f89c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92980b3cb0 con 0x7f9298159170 2023-06-23T18:35:59.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f929ce37700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92980ce620 0x7f9298144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f929ce37700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92980ce620 0x7f9298144820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39258/0 (socket says 172.21.15.79:39258) 2023-06-23T18:35:59.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f929ce37700 1 -- 172.21.15.79:0/2479944593 learned_addr learned my addr 172.21.15.79:0/2479944593 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f929d638700 1 --2- 172.21.15.79:0/2479944593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92980cd900 0x7f92980b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f929ce37700 1 -- 172.21.15.79:0/2479944593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9298159170 msgr2=0x7f929814e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f929ce37700 1 --2- 172.21.15.79:0/2479944593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9298159170 0x7f929814e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f929ce37700 1 -- 172.21.15.79:0/2479944593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92980cd900 msgr2=0x7f92980b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f929ce37700 1 --2- 172.21.15.79:0/2479944593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92980cd900 0x7f92980b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f929ce37700 1 -- 172.21.15.79:0/2479944593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9294009550 con 0x7f92980ce620 2023-06-23T18:35:59.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f929ce37700 1 --2- 172.21.15.79:0/2479944593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92980ce620 0x7f9298144820 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9294000c00 tx=0x7f9294004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7fd732d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd7340adce0 0x7fd734069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7fd73259c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd73415ccc0 0x7fd73406a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7fd732d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd7340adce0 0x7fd734069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39250/0 (socket says 172.21.15.79:39250) 2023-06-23T18:35:59.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.318+0000 7fd732d9d700 1 -- 172.21.15.79:0/210291588 learned_addr learned my addr 172.21.15.79:0/210291588 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7fd732d9d700 1 -- 172.21.15.79:0/210291588 >> v2:172.21.15.155:3300/0 conn(0x7fd73415ccc0 msgr2=0x7fd73406a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7fd732d9d700 1 --2- 172.21.15.79:0/210291588 >> v2:172.21.15.155:3300/0 conn(0x7fd73415ccc0 0x7fd73406a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7fd732d9d700 1 -- 172.21.15.79:0/210291588 >> v1:172.21.15.79:6789/0 conn(0x7fd734158a90 legacy=0x7fd734158ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:35:59.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7fd732d9d700 1 -- 172.21.15.79:0/210291588 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73415c4c0 con 0x7fd7340adce0 2023-06-23T18:35:59.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7fd732d9d700 1 --2- 172.21.15.79:0/210291588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7340adce0 0x7fd734069e90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd72400dd10 tx=0x7fd72400f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ffb090e146b432 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f548a531700 1 -- 172.21.15.79:0/3128058644 shutdown_connections 2023-06-23T18:35:59.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f548a531700 1 --2- 172.21.15.79:0/3128058644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54840c4dd0 0x7f548406a3d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f548a531700 1 --2- 172.21.15.79:0/3128058644 >> v2:172.21.15.79:3300/0 conn(0x7f54841608e0 0x7f5484069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f548a531700 1 -- 172.21.15.79:0/3128058644 >> 172.21.15.79:0/3128058644 conn(0x7f54840cb1c0 msgr2=0x7f54840cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f548a531700 1 -- 172.21.15.79:0/3128058644 shutdown_connections 2023-06-23T18:35:59.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f548a531700 1 -- 172.21.15.79:0/3128058644 wait complete. 2023-06-23T18:35:59.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7fd731d9b700 1 -- 172.21.15.79:0/210291588 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd724012070 con 0x7fd7340adce0 2023-06-23T18:35:59.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.320+0000 7fd731d9b700 1 -- 172.21.15.79:0/210291588 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72400d2e0 con 0x7fd7340adce0 2023-06-23T18:35:59.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.320+0000 7fd731d9b700 1 -- 172.21.15.79:0/210291588 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd724004230 con 0x7fd7340adce0 2023-06-23T18:35:59.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.320+0000 7fd739103700 1 -- 172.21.15.79:0/210291588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7340adce0 msgr2=0x7fd734069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.320+0000 7fd739103700 1 --2- 172.21.15.79:0/210291588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7340adce0 0x7fd734069e90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd72400dd10 tx=0x7fd72400f330 comp rx=0 tx=0).stop 2023-06-23T18:35:59.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.321+0000 7fd739103700 1 -- 172.21.15.79:0/210291588 shutdown_connections 2023-06-23T18:35:59.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.321+0000 7fd739103700 1 --2- 172.21.15.79:0/210291588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7340adce0 0x7fd734069e90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.321+0000 7fd739103700 1 --2- 172.21.15.79:0/210291588 >> v2:172.21.15.155:3300/0 conn(0x7fd73415ccc0 0x7fd73406a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.321+0000 7fd739103700 1 -- 172.21.15.79:0/210291588 >> 172.21.15.79:0/210291588 conn(0x7fd7340cb1c0 msgr2=0x7fd7340cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.321+0000 7fd739103700 1 -- 172.21.15.79:0/210291588 shutdown_connections 2023-06-23T18:35:59.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.321+0000 7fd739103700 1 -- 172.21.15.79:0/210291588 wait complete. 2023-06-23T18:35:59.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.321+0000 7f5ca5ffb700 1 -- 172.21.15.79:0/3701033040 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca000e070 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.321+0000 7f5ca5ffb700 1 -- 172.21.15.79:0/3701033040 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca0003a20 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.321+0000 7f5ca5ffb700 1 -- 172.21.15.79:0/3701033040 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0003dd0 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.321+0000 7f548a531700 1 Processor -- start 2023-06-23T18:35:59.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f928dffb700 1 -- 172.21.15.79:0/2479944593 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294004520 con 0x7f92980ce620 2023-06-23T18:35:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.319+0000 7f929f89c700 1 -- 172.21.15.79:0/2479944593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92980b3f90 con 0x7f92980ce620 2023-06-23T18:35:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.320+0000 7f929f89c700 1 -- 172.21.15.79:0/2479944593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92980b4550 con 0x7f92980ce620 2023-06-23T18:35:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.320+0000 7f928dffb700 1 -- 172.21.15.79:0/2479944593 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929401e070 con 0x7f92980ce620 2023-06-23T18:35:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.320+0000 7f928dffb700 1 -- 172.21.15.79:0/2479944593 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92940038c0 con 0x7f92980ce620 2023-06-23T18:35:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.321+0000 7f929f89c700 1 -- 172.21.15.79:0/2479944593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f929815d840 con 0x7f92980ce620 2023-06-23T18:35:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.322+0000 7f928dffb700 1 -- 172.21.15.79:0/2479944593 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9294003a20 con 0x7f92980ce620 2023-06-23T18:35:59.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.322+0000 7f3db597b700 1 -- 172.21.15.79:0/1636927561 shutdown_connections 2023-06-23T18:35:59.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.322+0000 7f3db597b700 1 --2- 172.21.15.79:0/1636927561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db015c4b0 0x7f3db00c3a00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.322+0000 7f3db597b700 1 -- 172.21.15.79:0/1636927561 >> 172.21.15.79:0/1636927561 conn(0x7f3db00cb1c0 msgr2=0x7f3db00cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.322+0000 7f5cada2b700 1 -- 172.21.15.79:0/3701033040 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca80c4dd0 msgr2=0x7f5ca806a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.322+0000 7f5cada2b700 1 --2- 172.21.15.79:0/3701033040 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca80c4dd0 0x7f5ca806a3d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0009e50 tx=0x7f5ca000b330 comp rx=0 tx=0).stop 2023-06-23T18:35:59.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.322+0000 7f3db597b700 1 -- 172.21.15.79:0/1636927561 shutdown_connections 2023-06-23T18:35:59.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.322+0000 7f3db597b700 1 -- 172.21.15.79:0/1636927561 wait complete. 2023-06-23T18:35:59.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.322+0000 7f548a531700 1 -- start start 2023-06-23T18:35:59.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7f5cada2b700 1 -- 172.21.15.79:0/3701033040 shutdown_connections 2023-06-23T18:35:59.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7f5cada2b700 1 --2- 172.21.15.79:0/3701033040 >> v2:172.21.15.79:3300/0 conn(0x7f5ca81608e0 0x7f5ca8069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7f5cada2b700 1 --2- 172.21.15.79:0/3701033040 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca80c4dd0 0x7f5ca806a3d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7f5cada2b700 1 -- 172.21.15.79:0/3701033040 >> 172.21.15.79:0/3701033040 conn(0x7f5ca80cb1c0 msgr2=0x7f5ca80cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7f5cada2b700 1 -- 172.21.15.79:0/3701033040 shutdown_connections 2023-06-23T18:35:59.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7fd739103700 1 Processor -- start 2023-06-23T18:35:59.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7f928dffb700 1 --2- 172.21.15.79:0/2479944593 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f927c06ace0 0x7f927c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7f928dffb700 1 -- 172.21.15.79:0/2479944593 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9294020070 con 0x7f92980ce620 2023-06-23T18:35:59.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7f928dffb700 1 --2- 172.21.15.79:0/2479944593 >> [v2:172.21.15.155:6800/4051222998,v1:172.21.15.155:6803/4051222998] conn(0x7f927c073490 0x7f927c0758a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7f928dffb700 1 -- 172.21.15.79:0/2479944593 --> [v2:172.21.15.155:6800/4051222998,v1:172.21.15.155:6803/4051222998] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f927c075f50 con 0x7f927c073490 2023-06-23T18:35:59.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7f929d638700 1 --2- 172.21.15.79:0/2479944593 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f927c06ace0 0x7f927c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7f548a531700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54840c4dd0 0x7f5484161d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7f548a531700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54840c52f0 0x7f54840b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f548a531700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54841608e0 0x7f54841610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f548a531700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54840b1bd0 con 0x7f54840c52f0 2023-06-23T18:35:59.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f548a531700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5484161600 con 0x7f54840c4dd0 2023-06-23T18:35:59.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f548a531700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54841629c0 con 0x7f54841608e0 2023-06-23T18:35:59.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7fd739103700 1 -- start start 2023-06-23T18:35:59.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7fd739103700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7340adce0 0x7fd7340c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7fd739103700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd734158a90 0x7fd7341542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7fd739103700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd73415ccc0 0x7fd7340b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7fd739103700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd734155940 con 0x7fd7340adce0 2023-06-23T18:35:59.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f928dffb700 1 -- 172.21.15.79:0/2479944593 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9294006ce0 con 0x7f92980ce620 2023-06-23T18:35:59.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7fd732d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7340adce0 0x7fd7340c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7fd732d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7340adce0 0x7fd7340c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39268/0 (socket says 172.21.15.79:39268) 2023-06-23T18:35:59.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7fd732d9d700 1 -- 172.21.15.79:0/655406181 learned_addr learned my addr 172.21.15.79:0/655406181 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f5488ace700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54841608e0 0x7f54841610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f929d638700 1 --2- 172.21.15.79:0/2479944593 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f927c06ace0 0x7f927c06d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9288009a10 tx=0x7f9288009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f5488ace700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54841608e0 0x7f54841610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:60266/0 (socket says 172.21.15.79:60266) 2023-06-23T18:35:59.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f5488ace700 1 -- 172.21.15.79:0/1073175239 learned_addr learned my addr 172.21.15.79:0/1073175239 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f3db597b700 1 Processor -- start 2023-06-23T18:35:59.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f929de39700 1 --2- 172.21.15.79:0/2479944593 >> [v2:172.21.15.155:6800/4051222998,v1:172.21.15.155:6803/4051222998] conn(0x7f927c073490 0x7f927c0758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.323+0000 7f5cada2b700 1 -- 172.21.15.79:0/3701033040 wait complete. 2023-06-23T18:35:59.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f54837fe700 1 --2- 172.21.15.79:0/1073175239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54840c52f0 0x7f54840b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f5488ace700 1 -- 172.21.15.79:0/1073175239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54840c4dd0 msgr2=0x7f5484161d00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-23T18:35:59.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f5488ace700 1 --2- 172.21.15.79:0/1073175239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54840c4dd0 0x7f5484161d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f5488ace700 1 -- 172.21.15.79:0/1073175239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54840c52f0 msgr2=0x7f54840b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f5488ace700 1 --2- 172.21.15.79:0/1073175239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54840c52f0 0x7f54840b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7f5488ace700 1 -- 172.21.15.79:0/1073175239 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c009b30 con 0x7f54841608e0 2023-06-23T18:35:59.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7f5cada2b700 1 Processor -- start 2023-06-23T18:35:59.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7f5488ace700 1 --2- 172.21.15.79:0/1073175239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54841608e0 0x7f54841610c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f547c00b370 tx=0x7f547c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7f929de39700 1 --2- 172.21.15.79:0/2479944593 >> [v2:172.21.15.155:6800/4051222998,v1:172.21.15.155:6803/4051222998] conn(0x7f927c073490 0x7f927c0758a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7f5cada2b700 1 -- start start 2023-06-23T18:35:59.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7fd73259c700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd734158a90 0x7fd7341542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7fd739103700 1 -- 172.21.15.79:0/655406181 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd734154cc0 con 0x7fd73415ccc0 2023-06-23T18:35:59.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.324+0000 7fd739103700 1 -- 172.21.15.79:0/655406181 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd734154fa0 con 0x7fd734158a90 2023-06-23T18:35:59.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7fd73359e700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd73415ccc0 0x7fd7340b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7fd73259c700 1 -- 172.21.15.79:0/655406181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd73415ccc0 msgr2=0x7fd7340b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7fd73259c700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd73415ccc0 0x7fd7340b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7fd73259c700 1 -- 172.21.15.79:0/655406181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7340adce0 msgr2=0x7fd7340c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7fd73259c700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7340adce0 0x7fd7340c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7fd73259c700 1 -- 172.21.15.79:0/655406181 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72400d9f0 con 0x7fd734158a90 2023-06-23T18:35:59.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7f5480ff9700 1 -- 172.21.15.79:0/1073175239 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c004970 con 0x7f54841608e0 2023-06-23T18:35:59.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7fd73359e700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd73415ccc0 0x7fd7340b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:35:59.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7f3db597b700 1 -- start start 2023-06-23T18:35:59.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7f548a531700 1 -- 172.21.15.79:0/1073175239 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5484162400 con 0x7f54841608e0 2023-06-23T18:35:59.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7f5480ff9700 1 -- 172.21.15.79:0/1073175239 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c0092e0 con 0x7f54841608e0 2023-06-23T18:35:59.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7f5480ff9700 1 -- 172.21.15.79:0/1073175239 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c01b670 con 0x7f54841608e0 2023-06-23T18:35:59.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7f548a531700 1 -- 172.21.15.79:0/1073175239 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54840afe20 con 0x7f54841608e0 2023-06-23T18:35:59.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f928dffb700 1 -- 172.21.15.79:0/2479944593 <== osd.5 v2:172.21.15.155:6800/4051222998 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f927c075f50 con 0x7f927c073490 2023-06-23T18:35:59.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f3db597b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db00c3f40 0x7f3db015cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f3db597b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3db015c4b0 0x7f3db01482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f3db597b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db015e150 0x7f3db0070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f3db597b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db015dcc0 con 0x7f3db015c4b0 2023-06-23T18:35:59.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f3db597b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db00aefd0 con 0x7f3db00c3f40 2023-06-23T18:35:59.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f3db597b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db015d0b0 con 0x7f3db015e150 2023-06-23T18:35:59.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f5cada2b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca80c4dd0 0x7f5ca80b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f5cada2b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca80c52f0 0x7f5ca80b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f5cada2b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca81608e0 0x7f5ca806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f5cada2b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca80b4560 con 0x7f5ca81608e0 2023-06-23T18:35:59.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f5cada2b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca80b3fa0 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f5cada2b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca80b4280 con 0x7f5ca80c52f0 2023-06-23T18:35:59.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f427b574700 1 Processor -- start 2023-06-23T18:35:59.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7fd73259c700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd734158a90 0x7fd7341542e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd728007ad0 tx=0x7fd728007e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.325+0000 7fd71b7fe700 1 -- 172.21.15.79:0/655406181 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728011040 con 0x7fd734158a90 2023-06-23T18:35:59.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7fd71b7fe700 1 -- 172.21.15.79:0/655406181 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7280088a0 con 0x7fd734158a90 2023-06-23T18:35:59.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7fd739103700 1 -- 172.21.15.79:0/655406181 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd734155280 con 0x7fd734158a90 2023-06-23T18:35:59.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7fd71b7fe700 1 -- 172.21.15.79:0/655406181 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728008c30 con 0x7fd734158a90 2023-06-23T18:35:59.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f3dae7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3db015c4b0 0x7f3db01482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f3dae7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3db015c4b0 0x7f3db01482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39278/0 (socket says 172.21.15.79:39278) 2023-06-23T18:35:59.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f3dae7fc700 1 -- 172.21.15.79:0/126849275 learned_addr learned my addr 172.21.15.79:0/126849275 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f5ca6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca80c4dd0 0x7f5ca80b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f5ca6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca80c4dd0 0x7f5ca80b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:45510/0 (socket says 172.21.15.79:45510) 2023-06-23T18:35:59.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f5ca6ffd700 1 -- 172.21.15.79:0/3337142661 learned_addr learned my addr 172.21.15.79:0/3337142661 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f5ca67fc700 1 --2- 172.21.15.79:0/3337142661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca80c52f0 0x7f5ca80b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7fd739103700 1 -- 172.21.15.79:0/655406181 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7340c83a0 con 0x7fd734158a90 2023-06-23T18:35:59.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f5ca6ffd700 1 -- 172.21.15.79:0/3337142661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca80c52f0 msgr2=0x7f5ca80b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f5ca6ffd700 1 --2- 172.21.15.79:0/3337142661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca80c52f0 0x7f5ca80b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f5ca6ffd700 1 -- 172.21.15.79:0/3337142661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca81608e0 msgr2=0x7f5ca806d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f5ca6ffd700 1 --2- 172.21.15.79:0/3337142661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca81608e0 0x7f5ca806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f5ca6ffd700 1 -- 172.21.15.79:0/3337142661 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca0009b30 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f5ca6ffd700 1 --2- 172.21.15.79:0/3337142661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca80c4dd0 0x7f5ca80b1b30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00b530 tx=0x7f5c9c00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f5c8ffff700 1 -- 172.21.15.79:0/3337142661 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c011830 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f5c8ffff700 1 -- 172.21.15.79:0/3337142661 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c0119b0 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f5c8ffff700 1 -- 172.21.15.79:0/3337142661 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c011d60 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f5cada2b700 1 -- 172.21.15.79:0/3337142661 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca80ae3a0 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f5cada2b700 1 -- 172.21.15.79:0/3337142661 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca80b2070 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f3dae7fc700 1 -- 172.21.15.79:0/126849275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db015e150 msgr2=0x7f3db0070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f3daeffd700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db00c3f40 0x7f3db015cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f3daf7fe700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db015e150 0x7f3db0070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f3dae7fc700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db015e150 0x7f3db0070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f3dae7fc700 1 -- 172.21.15.79:0/126849275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db00c3f40 msgr2=0x7f3db015cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f3dae7fc700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db00c3f40 0x7f3db015cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f3dae7fc700 1 -- 172.21.15.79:0/126849275 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da40094a0 con 0x7f3db015c4b0 2023-06-23T18:35:59.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.326+0000 7f427b574700 1 -- start start 2023-06-23T18:35:59.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f427b574700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4274160590 0x7f42740d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f427b574700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4274164280 0x7f42741646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f427b574700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4274164c00 con 0x7f4274164e90 2023-06-23T18:35:59.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f427b574700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42740cfee0 con 0x7f4274160590 2023-06-23T18:35:59.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.327+0000 7f427b574700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42740ce730 con 0x7f4274164280 2023-06-23T18:35:59.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f4279b11700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4274164280 0x7f42741646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f4279b11700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4274164280 0x7f42741646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:60292/0 (socket says 172.21.15.79:60292) 2023-06-23T18:35:59.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f4279b11700 1 -- 172.21.15.79:0/314638132 learned_addr learned my addr 172.21.15.79:0/314638132 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f4279310700 1 --2- 172.21.15.79:0/314638132 >> v2:172.21.15.79:3300/0 conn(0x7f4274160590 0x7f42740d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f4279b11700 1 -- 172.21.15.79:0/314638132 >> v2:172.21.15.79:3300/0 conn(0x7f4274160590 msgr2=0x7f42740d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f4279b11700 1 --2- 172.21.15.79:0/314638132 >> v2:172.21.15.79:3300/0 conn(0x7f4274160590 0x7f42740d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.388 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f4279b11700 1 -- 172.21.15.79:0/314638132 >> v1:172.21.15.79:6789/0 conn(0x7f4274164e90 legacy=0x7f42740cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.388 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f4279b11700 1 -- 172.21.15.79:0/314638132 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42740cea10 con 0x7f4274164280 2023-06-23T18:35:59.388 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f4279310700 1 --2- 172.21.15.79:0/314638132 >> v2:172.21.15.79:3300/0 conn(0x7f4274160590 0x7f42740d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:59.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.329+0000 7f4279b11700 1 --2- 172.21.15.79:0/314638132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4274164280 0x7f42741646c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f426c009e50 tx=0x7f426c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b04ed8edcfcf40d6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.329+0000 7f426bfff700 1 -- 172.21.15.79:0/314638132 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c00e070 con 0x7f4274164280 2023-06-23T18:35:59.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.329+0000 7f426bfff700 1 -- 172.21.15.79:0/314638132 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c003a20 con 0x7f4274164280 2023-06-23T18:35:59.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.329+0000 7f426bfff700 1 -- 172.21.15.79:0/314638132 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c003dd0 con 0x7f4274164280 2023-06-23T18:35:59.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.329+0000 7fd71b7fe700 1 -- 172.21.15.79:0/655406181 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd72800a5c0 con 0x7fd734158a90 2023-06-23T18:35:59.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f3dae7fc700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3db015c4b0 0x7f3db01482f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3da000c750 tx=0x7f3da000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f3daeffd700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db00c3f40 0x7f3db015cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:35:59.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f3d9ffff700 1 -- 172.21.15.79:0/126849275 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da0004c80 con 0x7f3db015c4b0 2023-06-23T18:35:59.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f3d9ffff700 1 -- 172.21.15.79:0/126849275 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da0004de0 con 0x7f3db015c4b0 2023-06-23T18:35:59.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.328+0000 7f3d9ffff700 1 -- 172.21.15.79:0/126849275 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da00072d0 con 0x7f3db015c4b0 2023-06-23T18:35:59.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.329+0000 7f3daf7fe700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db015e150 0x7f3db0070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:35:59.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.329+0000 7fd71b7fe700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd72006ace0 0x7fd72006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f427b574700 1 -- 172.21.15.79:0/314638132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4274164280 msgr2=0x7f42741646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f427b574700 1 --2- 172.21.15.79:0/314638132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4274164280 0x7f42741646c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f426c009e50 tx=0x7f426c00b330 comp rx=0 tx=0).stop 2023-06-23T18:35:59.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f427b574700 1 -- 172.21.15.79:0/314638132 shutdown_connections 2023-06-23T18:35:59.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f427b574700 1 --2- 172.21.15.79:0/314638132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4274164280 0x7f42741646c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f427b574700 1 --2- 172.21.15.79:0/314638132 >> v2:172.21.15.79:3300/0 conn(0x7f4274160590 0x7f42740d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f427b574700 1 -- 172.21.15.79:0/314638132 >> 172.21.15.79:0/314638132 conn(0x7f427406b350 msgr2=0x7f427406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.329+0000 7f548a531700 1 -- 172.21.15.79:0/1073175239 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5484160cb0 con 0x7f54841608e0 2023-06-23T18:35:59.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f5480ff9700 1 -- 172.21.15.79:0/1073175239 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f547c01b7d0 con 0x7f54841608e0 2023-06-23T18:35:59.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f427b574700 1 -- 172.21.15.79:0/314638132 shutdown_connections 2023-06-23T18:35:59.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f5cada2b700 1 -- 172.21.15.79:0/3337142661 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5ca8160cb0 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f5480ff9700 1 --2- 172.21.15.79:0/1073175239 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f547006adb0 0x7f547006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f5480ff9700 1 -- 172.21.15.79:0/1073175239 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f547c08aa30 con 0x7f54841608e0 2023-06-23T18:35:59.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f5480ff9700 1 --2- 172.21.15.79:0/1073175239 >> [v2:172.21.15.155:6824/3369626517,v1:172.21.15.155:6825/3369626517] conn(0x7f54700735e0 0x7f54700759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f5480ff9700 1 -- 172.21.15.79:0/1073175239 --> [v2:172.21.15.155:6824/3369626517,v1:172.21.15.155:6825/3369626517] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54700760a0 con 0x7f54700735e0 2023-06-23T18:35:59.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f3db597b700 1 -- 172.21.15.79:0/126849275 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db015d390 con 0x7f3db015c4b0 2023-06-23T18:35:59.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.331+0000 7f3db597b700 1 -- 172.21.15.79:0/126849275 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db015d950 con 0x7f3db015c4b0 2023-06-23T18:35:59.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7f427b574700 1 -- 172.21.15.79:0/314638132 wait complete. 2023-06-23T18:35:59.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7fd71b7fe700 1 -- 172.21.15.79:0/655406181 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd7280893d0 con 0x7fd734158a90 2023-06-23T18:35:59.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7fd732d9d700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd72006ace0 0x7fd72006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7fd739103700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.155:6801/1282622313,v1:172.21.15.155:6802/1282622313] conn(0x7fd7140015e0 0x7fd714003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7fd739103700 1 -- 172.21.15.79:0/655406181 --> [v2:172.21.15.155:6801/1282622313,v1:172.21.15.155:6802/1282622313] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd714006bc0 con 0x7fd7140015e0 2023-06-23T18:35:59.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.330+0000 7fd73359e700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.155:6801/1282622313,v1:172.21.15.155:6802/1282622313] conn(0x7fd7140015e0 0x7fd714003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.331+0000 7f5483fff700 1 --2- 172.21.15.79:0/1073175239 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f547006adb0 0x7f547006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.331+0000 7f54837fe700 1 --2- 172.21.15.79:0/1073175239 >> [v2:172.21.15.155:6824/3369626517,v1:172.21.15.155:6825/3369626517] conn(0x7f54700735e0 0x7f54700759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.331+0000 7f5c8ffff700 1 -- 172.21.15.79:0/3337142661 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c9c01a400 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.331+0000 7f159343e700 1 Processor -- start 2023-06-23T18:35:59.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.331+0000 7f159343e700 1 -- start start 2023-06-23T18:35:59.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.331+0000 7fd73359e700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.155:6801/1282622313,v1:172.21.15.155:6802/1282622313] conn(0x7fd7140015e0 0x7fd714003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.331+0000 7f3db597b700 1 -- 172.21.15.79:0/126849275 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3db015c880 con 0x7f3db015c4b0 2023-06-23T18:35:59.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.332+0000 7fd732d9d700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd72006ace0 0x7fd72006d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd724000c00 tx=0x7fd72400d000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.332+0000 7f54837fe700 1 --2- 172.21.15.79:0/1073175239 >> [v2:172.21.15.155:6824/3369626517,v1:172.21.15.155:6825/3369626517] conn(0x7f54700735e0 0x7f54700759f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.332+0000 7f5483fff700 1 --2- 172.21.15.79:0/1073175239 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f547006adb0 0x7f547006d260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5478009a70 tx=0x7f5478009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.332+0000 7f5c8ffff700 1 --2- 172.21.15.79:0/3337142661 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5c9406ace0 0x7f5c9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.332+0000 7f5ca67fc700 1 --2- 172.21.15.79:0/3337142661 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5c9406ace0 0x7f5c9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.332+0000 7f5c8ffff700 1 -- 172.21.15.79:0/3337142661 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c9c089490 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.332+0000 7f5c8ffff700 1 --2- 172.21.15.79:0/3337142661 >> [v2:172.21.15.79:6800/2920568506,v1:172.21.15.79:6801/2920568506] conn(0x7f5c94073490 0x7f5c940758a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.332+0000 7f5c8ffff700 1 -- 172.21.15.79:0/3337142661 --> [v2:172.21.15.79:6800/2920568506,v1:172.21.15.79:6801/2920568506] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c94075f50 con 0x7f5c94073490 2023-06-23T18:35:59.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.332+0000 7f159343e700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f158c160590 0x7f158c0d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.332+0000 7f159343e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f158c164e90 0x7f158c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.332+0000 7f159343e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c0ce730 con 0x7f158c0cfd10 2023-06-23T18:35:59.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.332+0000 7f159343e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c0cea10 con 0x7f158c164e90 2023-06-23T18:35:59.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.332+0000 7f159343e700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c0cecf0 con 0x7f158c160590 2023-06-23T18:35:59.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.332+0000 7f427b574700 1 Processor -- start 2023-06-23T18:35:59.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.333+0000 7fd71b7fe700 1 -- 172.21.15.79:0/655406181 <== osd.6 v2:172.21.15.155:6801/1282622313 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd714006bc0 con 0x7fd7140015e0 2023-06-23T18:35:59.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.333+0000 7f5480ff9700 1 -- 172.21.15.79:0/1073175239 <== osd.7 v2:172.21.15.155:6824/3369626517 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f54700760a0 con 0x7f54700735e0 2023-06-23T18:35:59.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.333+0000 7f15911da700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f158c160590 0x7f158c0d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.333+0000 7f15911da700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f158c160590 0x7f158c0d10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:45532/0 (socket says 172.21.15.79:45532) 2023-06-23T18:35:59.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.333+0000 7f15911da700 1 -- 172.21.15.79:0/706752068 learned_addr learned my addr 172.21.15.79:0/706752068 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.333+0000 7f5ca77fe700 1 --2- 172.21.15.79:0/3337142661 >> [v2:172.21.15.79:6800/2920568506,v1:172.21.15.79:6801/2920568506] conn(0x7f5c94073490 0x7f5c940758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.333+0000 7f427b574700 1 -- start start 2023-06-23T18:35:59.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.333+0000 7f5480ff9700 1 -- 172.21.15.79:0/1073175239 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f547c0099b0 con 0x7f54841608e0 2023-06-23T18:35:59.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.333+0000 7f15911da700 1 -- 172.21.15.79:0/706752068 >> v2:172.21.15.79:3300/0 conn(0x7f158c164e90 msgr2=0x7f158c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.333+0000 7f15911da700 1 --2- 172.21.15.79:0/706752068 >> v2:172.21.15.79:3300/0 conn(0x7f158c164e90 0x7f158c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.333+0000 7f15911da700 1 -- 172.21.15.79:0/706752068 >> v1:172.21.15.79:6789/0 conn(0x7f158c0cfd10 legacy=0x7f158c164280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.333+0000 7f15911da700 1 -- 172.21.15.79:0/706752068 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c0cefd0 con 0x7f158c160590 2023-06-23T18:35:59.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.333+0000 7f5ca77fe700 1 --2- 172.21.15.79:0/3337142661 >> [v2:172.21.15.79:6800/2920568506,v1:172.21.15.79:6801/2920568506] conn(0x7f5c94073490 0x7f5c940758a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f427b574700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4274160590 0x7f42740b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f427b574700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4274164280 0x7f42740cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f427b574700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4274164e90 0x7f42740ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f427b574700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42740c8990 con 0x7f4274160590 2023-06-23T18:35:59.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f427b574700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427415a0f0 con 0x7f4274164e90 2023-06-23T18:35:59.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f427b574700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427415a3d0 con 0x7f4274164280 2023-06-23T18:35:59.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f4279310700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4274160590 0x7f42740b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f4279310700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4274160590 0x7f42740b7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39312/0 (socket says 172.21.15.79:39312) 2023-06-23T18:35:59.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f4279310700 1 -- 172.21.15.79:0/1130067924 learned_addr learned my addr 172.21.15.79:0/1130067924 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f4278b0f700 1 --2- 172.21.15.79:0/1130067924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4274164280 0x7f42740cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f4279b11700 1 --2- 172.21.15.79:0/1130067924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4274164e90 0x7f42740ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f4279310700 1 -- 172.21.15.79:0/1130067924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4274164280 msgr2=0x7f42740cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f4279310700 1 --2- 172.21.15.79:0/1130067924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4274164280 0x7f42740cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f4279310700 1 -- 172.21.15.79:0/1130067924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4274164e90 msgr2=0x7f42740ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f4279310700 1 --2- 172.21.15.79:0/1130067924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4274164e90 0x7f42740ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f4279310700 1 -- 172.21.15.79:0/1130067924 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42700094a0 con 0x7f4274160590 2023-06-23T18:35:59.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f4279310700 1 --2- 172.21.15.79:0/1130067924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4274160590 0x7f42740b7bc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f427000ebe0 tx=0x7f427000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f15911da700 1 --2- 172.21.15.79:0/706752068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f158c160590 0x7f158c0d10f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f158c0d3be0 tx=0x7f157c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a34df6ac99c64810 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f3d9ffff700 1 -- 172.21.15.79:0/126849275 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3da0007430 con 0x7f3db015c4b0 2023-06-23T18:35:59.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f3d9ffff700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3d9406acb0 0x7f3d9406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f3d9ffff700 1 -- 172.21.15.79:0/126849275 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3da0089bc0 con 0x7f3db015c4b0 2023-06-23T18:35:59.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f3daeffd700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3d9406acb0 0x7f3d9406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.334+0000 7f3d9ffff700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.79:6808/1517442919,v1:172.21.15.79:6809/1517442919] conn(0x7f3d940734e0 0x7f3d940758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.335+0000 7f3d9ffff700 1 -- 172.21.15.79:0/126849275 --> [v2:172.21.15.79:6808/1517442919,v1:172.21.15.79:6809/1517442919] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d94075fa0 con 0x7f3d940734e0 2023-06-23T18:35:59.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.335+0000 7f3d9ffff700 1 -- 172.21.15.79:0/126849275 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3da0089f70 con 0x7f3db015c4b0 2023-06-23T18:35:59.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.335+0000 7f3daf7fe700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.79:6808/1517442919,v1:172.21.15.79:6809/1517442919] conn(0x7f3d940734e0 0x7f3d940758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.335+0000 7f5ca67fc700 1 --2- 172.21.15.79:0/3337142661 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5c9406ace0 0x7f5c9406d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8154c00 tx=0x7f5c98009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.335+0000 7f3daeffd700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3d9406acb0 0x7f3d9406d160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3db0152ce0 tx=0x7f3da4010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.335+0000 7f1583fff700 1 -- 172.21.15.79:0/706752068 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c00b870 con 0x7f158c160590 2023-06-23T18:35:59.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.335+0000 7f3daf7fe700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.79:6808/1517442919,v1:172.21.15.79:6809/1517442919] conn(0x7f3d940734e0 0x7f3d940758f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.335+0000 7f1583fff700 1 -- 172.21.15.79:0/706752068 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c00b9d0 con 0x7f158c160590 2023-06-23T18:35:59.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.335+0000 7f1583fff700 1 -- 172.21.15.79:0/706752068 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c00bd60 con 0x7f158c160590 2023-06-23T18:35:59.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.335+0000 7f4269ffb700 1 -- 172.21.15.79:0/1130067924 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427000c850 con 0x7f4274160590 2023-06-23T18:35:59.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.335+0000 7f4269ffb700 1 -- 172.21.15.79:0/1130067924 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427000c9b0 con 0x7f4274160590 2023-06-23T18:35:59.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.335+0000 7f4269ffb700 1 -- 172.21.15.79:0/1130067924 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427000cd40 con 0x7f4274160590 2023-06-23T18:35:59.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.336+0000 7f3d9ffff700 1 -- 172.21.15.79:0/126849275 <== osd.0 v2:172.21.15.79:6808/1517442919 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3d94075fa0 con 0x7f3d940734e0 2023-06-23T18:35:59.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.336+0000 7f159343e700 1 -- 172.21.15.79:0/706752068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f158c160590 msgr2=0x7f158c0d10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.336+0000 7f159343e700 1 --2- 172.21.15.79:0/706752068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f158c160590 0x7f158c0d10f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f158c0d3be0 tx=0x7f157c009bd0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.336+0000 7f159343e700 1 -- 172.21.15.79:0/706752068 shutdown_connections 2023-06-23T18:35:59.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.336+0000 7f159343e700 1 --2- 172.21.15.79:0/706752068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f158c160590 0x7f158c0d10f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.336+0000 7f159343e700 1 --2- 172.21.15.79:0/706752068 >> v2:172.21.15.79:3300/0 conn(0x7f158c164e90 0x7f158c0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.336+0000 7f159343e700 1 -- 172.21.15.79:0/706752068 >> 172.21.15.79:0/706752068 conn(0x7f158c06b350 msgr2=0x7f158c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.336+0000 7f427b574700 1 -- 172.21.15.79:0/1130067924 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426c009b30 con 0x7f4274160590 2023-06-23T18:35:59.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.337+0000 7f427b574700 1 -- 172.21.15.79:0/1130067924 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427415a990 con 0x7f4274160590 2023-06-23T18:35:59.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.335+0000 7fd7ea53d700 1 Processor -- start 2023-06-23T18:35:59.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.335+0000 7fd7ea53d700 1 -- start start 2023-06-23T18:35:59.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.336+0000 7fd7ea53d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e41608e0 0x7fd7e4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.336+0000 7fd7ea53d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd7e40c4dd0 0x7fd7e406a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.336+0000 7fd7ea53d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e4164890 con 0x7fd7e40c52f0 2023-06-23T18:35:59.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.337+0000 7fd7ea53d700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e40c91c0 con 0x7fd7e40c4dd0 2023-06-23T18:35:59.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.337+0000 7fd7ea53d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e4164b70 con 0x7fd7e41608e0 2023-06-23T18:35:59.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.337+0000 7fd7e37fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd7e40c52f0 0x7fd7e4070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42364/0 (socket says 172.21.15.79:42364) 2023-06-23T18:35:59.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.337+0000 7fd7e37fe700 1 -- 172.21.15.79:0/3293971758 learned_addr learned my addr 172.21.15.79:0/3293971758 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:59.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.337+0000 7fd7e8ada700 1 --2- 172.21.15.79:0/3293971758 >> v2:172.21.15.155:3300/0 conn(0x7fd7e40c4dd0 0x7fd7e406a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.337+0000 7fd7e8ada700 1 -- 172.21.15.79:0/3293971758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e41608e0 msgr2=0x7fd7e4069e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:35:59.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.337+0000 7fd7e8ada700 1 --2- 172.21.15.79:0/3293971758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e41608e0 0x7fd7e4069e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.337+0000 7fd7e8ada700 1 -- 172.21.15.79:0/3293971758 >> v1:172.21.15.79:6789/0 conn(0x7fd7e40c52f0 legacy=0x7fd7e4070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.337+0000 7fd7e8ada700 1 -- 172.21.15.79:0/3293971758 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e4164e50 con 0x7fd7e40c4dd0 2023-06-23T18:35:59.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.338+0000 7fd7e8ada700 1 --2- 172.21.15.79:0/3293971758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e40c4dd0 0x7fd7e406a3d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009d90 tx=0x7fd7dc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2adca516cc42eb44 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.338+0000 7fd7e2ffd700 1 -- 172.21.15.79:0/3293971758 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc00e070 con 0x7fd7e40c4dd0 2023-06-23T18:35:59.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.338+0000 7fd7e2ffd700 1 -- 172.21.15.79:0/3293971758 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7dc00bbd0 con 0x7fd7e40c4dd0 2023-06-23T18:35:59.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.338+0000 7fd7e2ffd700 1 -- 172.21.15.79:0/3293971758 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc012430 con 0x7fd7e40c4dd0 2023-06-23T18:35:59.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.338+0000 7f4269ffb700 1 -- 172.21.15.79:0/1130067924 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4270013470 con 0x7f4274160590 2023-06-23T18:35:59.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.339+0000 7f4269ffb700 1 --2- 172.21.15.79:0/1130067924 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f425806ad30 0x7f425806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.339+0000 7fd7ea53d700 1 -- 172.21.15.79:0/3293971758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e40c4dd0 msgr2=0x7fd7e406a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.339+0000 7fd7ea53d700 1 --2- 172.21.15.79:0/3293971758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e40c4dd0 0x7fd7e406a3d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009d90 tx=0x7fd7dc00b330 comp rx=0 tx=0).stop 2023-06-23T18:35:59.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.339+0000 7f4269ffb700 1 -- 172.21.15.79:0/1130067924 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4270089420 con 0x7f4274160590 2023-06-23T18:35:59.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.339+0000 7f4278b0f700 1 --2- 172.21.15.79:0/1130067924 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f425806ad30 0x7f425806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.339+0000 7f4278b0f700 1 --2- 172.21.15.79:0/1130067924 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f425806ad30 0x7f425806d1e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f42740c9d50 tx=0x7f42640090e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.342+0000 7f5c8ffff700 1 -- 172.21.15.79:0/3337142661 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c9c009380 con 0x7f5ca80c4dd0 2023-06-23T18:35:59.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.342+0000 7f5c8ffff700 1 -- 172.21.15.79:0/3337142661 <== osd.1 v2:172.21.15.79:6800/2920568506 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5c94075f50 con 0x7f5c94073490 2023-06-23T18:35:59.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.342+0000 7f159343e700 1 -- 172.21.15.79:0/706752068 shutdown_connections 2023-06-23T18:35:59.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.344+0000 7fd7ea53d700 1 -- 172.21.15.79:0/3293971758 shutdown_connections 2023-06-23T18:35:59.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.344+0000 7fd7ea53d700 1 --2- 172.21.15.79:0/3293971758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e40c4dd0 0x7fd7e406a3d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.344+0000 7fd7ea53d700 1 --2- 172.21.15.79:0/3293971758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e41608e0 0x7fd7e4069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.344+0000 7fd7ea53d700 1 -- 172.21.15.79:0/3293971758 >> 172.21.15.79:0/3293971758 conn(0x7fd7e40cb1c0 msgr2=0x7fd7e40cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.344+0000 7f427b574700 1 --2- 172.21.15.79:0/1130067924 >> [v2:172.21.15.155:6816/2160815572,v1:172.21.15.155:6817/2160815572] conn(0x7f427415abb0 0x7f4274153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.344+0000 7f427b574700 1 -- 172.21.15.79:0/1130067924 --> [v2:172.21.15.155:6816/2160815572,v1:172.21.15.155:6817/2160815572] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42740cecf0 con 0x7f427415abb0 2023-06-23T18:35:59.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.342+0000 7f159343e700 1 -- 172.21.15.79:0/706752068 wait complete. 2023-06-23T18:35:59.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.343+0000 7f159343e700 1 Processor -- start 2023-06-23T18:35:59.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.343+0000 7f159343e700 1 -- start start 2023-06-23T18:35:59.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.344+0000 7f159343e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f158c160590 0x7f158c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.344+0000 7f159343e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f158c164e90 0x7f158c0c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.344+0000 7f159343e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f158c0c90c0 0x7f158c0c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.344+0000 7f159343e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c158080 con 0x7f158c164e90 2023-06-23T18:35:59.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.344+0000 7f159343e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0c9520 con 0x7f158c160590 2023-06-23T18:35:59.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.344+0000 7f159343e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c155e00 con 0x7f158c0c90c0 2023-06-23T18:35:59.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.345+0000 7f15919db700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f158c0c90c0 0x7f158c0c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.345+0000 7f15919db700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f158c0c90c0 0x7f158c0c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:60332/0 (socket says 172.21.15.79:60332) 2023-06-23T18:35:59.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.345+0000 7f15919db700 1 -- 172.21.15.79:0/2690148771 learned_addr learned my addr 172.21.15.79:0/2690148771 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.345+0000 7f15919db700 1 -- 172.21.15.79:0/2690148771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f158c160590 msgr2=0x7f158c0cbea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T18:35:59.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.345+0000 7f15919db700 1 --2- 172.21.15.79:0/2690148771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f158c160590 0x7f158c0cbea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.345+0000 7f15919db700 1 -- 172.21.15.79:0/2690148771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f158c164e90 msgr2=0x7f158c0c8b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T18:35:59.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.345+0000 7f15919db700 1 --2- 172.21.15.79:0/2690148771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f158c164e90 0x7f158c0c8b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.345+0000 7f15919db700 1 -- 172.21.15.79:0/2690148771 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c009550 con 0x7f158c0c90c0 2023-06-23T18:35:59.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.345+0000 7f15919db700 1 --2- 172.21.15.79:0/2690148771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f158c0c90c0 0x7f158c0c6910 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f158400cbd0 tx=0x7f158400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.345+0000 7f1581ffb700 1 -- 172.21.15.79:0/2690148771 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1584003c00 con 0x7f158c0c90c0 2023-06-23T18:35:59.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.345+0000 7f1581ffb700 1 -- 172.21.15.79:0/2690148771 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1584003d60 con 0x7f158c0c90c0 2023-06-23T18:35:59.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.345+0000 7f159343e700 1 -- 172.21.15.79:0/2690148771 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158c0c76c0 con 0x7f158c0c90c0 2023-06-23T18:35:59.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.345+0000 7f1581ffb700 1 -- 172.21.15.79:0/2690148771 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158400abb0 con 0x7f158c0c90c0 2023-06-23T18:35:59.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.346+0000 7f159343e700 1 -- 172.21.15.79:0/2690148771 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158c0c7f00 con 0x7f158c0c90c0 2023-06-23T18:35:59.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.346+0000 7f159343e700 1 -- 172.21.15.79:0/2690148771 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f158c0cf2b0 con 0x7f158c0c90c0 2023-06-23T18:35:59.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.347+0000 7f1581ffb700 1 -- 172.21.15.79:0/2690148771 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f158400ad10 con 0x7f158c0c90c0 2023-06-23T18:35:59.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.348+0000 7f1581ffb700 1 --2- 172.21.15.79:0/2690148771 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f157006ace0 0x7f157006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.348+0000 7f1581ffb700 1 -- 172.21.15.79:0/2690148771 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1584089590 con 0x7f158c0c90c0 2023-06-23T18:35:59.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.348+0000 7f1581ffb700 1 --2- 172.21.15.79:0/2690148771 >> [v2:172.21.15.79:6816/1735841648,v1:172.21.15.79:6817/1735841648] conn(0x7f1570073510 0x7f1570075920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.348+0000 7f1581ffb700 1 -- 172.21.15.79:0/2690148771 --> [v2:172.21.15.79:6816/1735841648,v1:172.21.15.79:6817/1735841648] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1570075fd0 con 0x7f1570073510 2023-06-23T18:35:59.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.348+0000 7f1581ffb700 1 -- 172.21.15.79:0/2690148771 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f158408bc50 con 0x7f158c0c90c0 2023-06-23T18:35:59.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.352+0000 7f15909d9700 1 --2- 172.21.15.79:0/2690148771 >> [v2:172.21.15.79:6816/1735841648,v1:172.21.15.79:6817/1735841648] conn(0x7f1570073510 0x7f1570075920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.352+0000 7f15911da700 1 --2- 172.21.15.79:0/2690148771 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f157006ace0 0x7f157006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.352+0000 7f4279b11700 1 --2- 172.21.15.79:0/1130067924 >> [v2:172.21.15.155:6816/2160815572,v1:172.21.15.155:6817/2160815572] conn(0x7f427415abb0 0x7f4274153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.352+0000 7f15909d9700 1 --2- 172.21.15.79:0/2690148771 >> [v2:172.21.15.79:6816/1735841648,v1:172.21.15.79:6817/1735841648] conn(0x7f1570073510 0x7f1570075920 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.353+0000 7f15911da700 1 --2- 172.21.15.79:0/2690148771 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f157006ace0 0x7f157006d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f157c009c10 tx=0x7f157c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.353+0000 7f1581ffb700 1 -- 172.21.15.79:0/2690148771 <== osd.2 v2:172.21.15.79:6816/1735841648 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1570075fd0 con 0x7f1570073510 2023-06-23T18:35:59.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.356+0000 7f548a531700 1 -- 172.21.15.79:0/1073175239 --> [v2:172.21.15.155:6824/3369626517,v1:172.21.15.155:6825/3369626517] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5484165130 con 0x7f54700735e0 2023-06-23T18:35:59.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.356+0000 7f4279b11700 1 --2- 172.21.15.79:0/1130067924 >> [v2:172.21.15.155:6816/2160815572,v1:172.21.15.155:6817/2160815572] conn(0x7f427415abb0 0x7f4274153ac0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.356+0000 7f5480ff9700 1 -- 172.21.15.79:0/1073175239 <== osd.7 v2:172.21.15.155:6824/3369626517 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5474001550 con 0x7f54700735e0 2023-06-23T18:35:59.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.357+0000 7f548a531700 1 -- 172.21.15.79:0/1073175239 >> [v2:172.21.15.155:6824/3369626517,v1:172.21.15.155:6825/3369626517] conn(0x7f54700735e0 msgr2=0x7f54700759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.357+0000 7f548a531700 1 --2- 172.21.15.79:0/1073175239 >> [v2:172.21.15.155:6824/3369626517,v1:172.21.15.155:6825/3369626517] conn(0x7f54700735e0 0x7f54700759f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.357+0000 7f548a531700 1 -- 172.21.15.79:0/1073175239 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f547006adb0 msgr2=0x7f547006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.357+0000 7f548a531700 1 --2- 172.21.15.79:0/1073175239 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f547006adb0 0x7f547006d260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5478009a70 tx=0x7f5478009210 comp rx=0 tx=0).stop 2023-06-23T18:35:59.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.357+0000 7f548a531700 1 -- 172.21.15.79:0/1073175239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54841608e0 msgr2=0x7f54841610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.357+0000 7f548a531700 1 --2- 172.21.15.79:0/1073175239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54841608e0 0x7f54841610c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f547c00b370 tx=0x7f547c004ad0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.357+0000 7f5488ace700 1 -- 172.21.15.79:0/1073175239 reap_dead start 2023-06-23T18:35:59.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.357+0000 7f548a531700 1 -- 172.21.15.79:0/1073175239 shutdown_connections 2023-06-23T18:35:59.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.357+0000 7f548a531700 1 -- 172.21.15.79:0/1073175239 >> 172.21.15.79:0/1073175239 conn(0x7f54840cb1c0 msgr2=0x7f54840cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.357+0000 7f548a531700 1 -- 172.21.15.79:0/1073175239 shutdown_connections 2023-06-23T18:35:59.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.357+0000 7f548a531700 1 -- 172.21.15.79:0/1073175239 wait complete. 2023-06-23T18:35:59.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.358+0000 7f929f89c700 1 -- 172.21.15.79:0/2479944593 --> [v2:172.21.15.155:6800/4051222998,v1:172.21.15.155:6803/4051222998] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f929815d560 con 0x7f927c073490 2023-06-23T18:35:59.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.358+0000 7f4269ffb700 1 -- 172.21.15.79:0/1130067924 <== osd.4 v2:172.21.15.155:6816/2160815572 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f426c003d90 con 0x7f427415abb0 2023-06-23T18:35:59.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.358+0000 7f928dffb700 1 -- 172.21.15.79:0/2479944593 <== osd.5 v2:172.21.15.155:6800/4051222998 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9290009ad0 con 0x7f927c073490 2023-06-23T18:35:59.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.360+0000 7f929f89c700 1 -- 172.21.15.79:0/2479944593 >> [v2:172.21.15.155:6800/4051222998,v1:172.21.15.155:6803/4051222998] conn(0x7f927c073490 msgr2=0x7f927c0758a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.360+0000 7f929f89c700 1 --2- 172.21.15.79:0/2479944593 >> [v2:172.21.15.155:6800/4051222998,v1:172.21.15.155:6803/4051222998] conn(0x7f927c073490 0x7f927c0758a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.360+0000 7f929f89c700 1 -- 172.21.15.79:0/2479944593 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f927c06ace0 msgr2=0x7f927c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.360+0000 7f929f89c700 1 --2- 172.21.15.79:0/2479944593 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f927c06ace0 0x7f927c06d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9288009a10 tx=0x7f9288009190 comp rx=0 tx=0).stop 2023-06-23T18:35:59.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.360+0000 7f929f89c700 1 -- 172.21.15.79:0/2479944593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92980ce620 msgr2=0x7f9298144820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.360+0000 7f929f89c700 1 --2- 172.21.15.79:0/2479944593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92980ce620 0x7f9298144820 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9294000c00 tx=0x7f9294004750 comp rx=0 tx=0).stop 2023-06-23T18:35:59.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.360+0000 7f929de39700 1 -- 172.21.15.79:0/2479944593 reap_dead start 2023-06-23T18:35:59.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.359+0000 7fd7ea53d700 1 -- 172.21.15.79:0/3293971758 shutdown_connections 2023-06-23T18:35:59.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.360+0000 7f929f89c700 1 -- 172.21.15.79:0/2479944593 shutdown_connections 2023-06-23T18:35:59.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.360+0000 7f929f89c700 1 -- 172.21.15.79:0/2479944593 >> 172.21.15.79:0/2479944593 conn(0x7f929806d660 msgr2=0x7f92980af130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.360+0000 7f929f89c700 1 -- 172.21.15.79:0/2479944593 shutdown_connections 2023-06-23T18:35:59.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.360+0000 7f929f89c700 1 -- 172.21.15.79:0/2479944593 wait complete. 2023-06-23T18:35:59.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.365+0000 7fd739103700 1 -- 172.21.15.79:0/655406181 --> [v2:172.21.15.155:6801/1282622313,v1:172.21.15.155:6802/1282622313] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd714005ca0 con 0x7fd7140015e0 2023-06-23T18:35:59.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.365+0000 7fd7ea53d700 1 -- 172.21.15.79:0/3293971758 wait complete. 2023-06-23T18:35:59.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.366+0000 7f3db597b700 1 -- 172.21.15.79:0/126849275 --> [v2:172.21.15.79:6808/1517442919,v1:172.21.15.79:6809/1517442919] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3db00c99b0 con 0x7f3d940734e0 2023-06-23T18:35:59.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.367+0000 7fd7ea53d700 1 Processor -- start 2023-06-23T18:35:59.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.367+0000 7fd7ea53d700 1 -- start start 2023-06-23T18:35:59.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.367+0000 7fd71b7fe700 1 -- 172.21.15.79:0/655406181 <== osd.6 v2:172.21.15.155:6801/1282622313 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd714005ca0 con 0x7fd7140015e0 2023-06-23T18:35:59.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.367+0000 7f3d9ffff700 1 -- 172.21.15.79:0/126849275 <== osd.0 v2:172.21.15.79:6808/1517442919 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3da8001e70 con 0x7f3d940734e0 2023-06-23T18:35:59.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.368+0000 7fd7ea53d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e40c4dd0 0x7fd7e40c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.368+0000 7fd7ea53d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e40c52f0 0x7fd7e40aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.368+0000 7fd7ea53d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e41608e0 0x7fd7e42024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.368+0000 7fd7ea53d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e40b08f0 con 0x7fd7e40c4dd0 2023-06-23T18:35:59.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.368+0000 7fd7ea53d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e40af7b0 con 0x7fd7e41608e0 2023-06-23T18:35:59.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.368+0000 7fd7ea53d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e414fea0 con 0x7fd7e40c52f0 2023-06-23T18:35:59.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.368+0000 7fd739103700 1 -- 172.21.15.79:0/655406181 >> [v2:172.21.15.155:6801/1282622313,v1:172.21.15.155:6802/1282622313] conn(0x7fd7140015e0 msgr2=0x7fd714003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.368+0000 7fd739103700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.155:6801/1282622313,v1:172.21.15.155:6802/1282622313] conn(0x7fd7140015e0 0x7fd714003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd7e3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e40c4dd0 0x7fd7e40c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd7e3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e40c4dd0 0x7fd7e40c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39338/0 (socket says 172.21.15.79:39338) 2023-06-23T18:35:59.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd7e3fff700 1 -- 172.21.15.79:0/2658829691 learned_addr learned my addr 172.21.15.79:0/2658829691 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd7e8ada700 1 --2- 172.21.15.79:0/2658829691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e41608e0 0x7fd7e42024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd7e37fe700 1 --2- 172.21.15.79:0/2658829691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e40c52f0 0x7fd7e40aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd7e37fe700 1 -- 172.21.15.79:0/2658829691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e41608e0 msgr2=0x7fd7e42024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd7e37fe700 1 --2- 172.21.15.79:0/2658829691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e41608e0 0x7fd7e42024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd7e37fe700 1 -- 172.21.15.79:0/2658829691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e40c4dd0 msgr2=0x7fd7e40c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd7e37fe700 1 --2- 172.21.15.79:0/2658829691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e40c4dd0 0x7fd7e40c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd7e37fe700 1 -- 172.21.15.79:0/2658829691 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc009a70 con 0x7fd7e40c52f0 2023-06-23T18:35:59.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd739103700 1 -- 172.21.15.79:0/655406181 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd72006ace0 msgr2=0x7fd72006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd739103700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd72006ace0 0x7fd72006d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd724000c00 tx=0x7fd72400d000 comp rx=0 tx=0).stop 2023-06-23T18:35:59.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd739103700 1 -- 172.21.15.79:0/655406181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd734158a90 msgr2=0x7fd7341542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd739103700 1 --2- 172.21.15.79:0/655406181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd734158a90 0x7fd7341542e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd728007ad0 tx=0x7fd728007e90 comp rx=0 tx=0).stop 2023-06-23T18:35:59.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd73359e700 1 -- 172.21.15.79:0/655406181 reap_dead start 2023-06-23T18:35:59.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7f3db597b700 1 -- 172.21.15.79:0/126849275 >> [v2:172.21.15.79:6808/1517442919,v1:172.21.15.79:6809/1517442919] conn(0x7f3d940734e0 msgr2=0x7f3d940758f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7f3db597b700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.79:6808/1517442919,v1:172.21.15.79:6809/1517442919] conn(0x7f3d940734e0 0x7f3d940758f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7f3db597b700 1 -- 172.21.15.79:0/126849275 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3d9406acb0 msgr2=0x7f3d9406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7f3db597b700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3d9406acb0 0x7f3d9406d160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3db0152ce0 tx=0x7f3da4010000 comp rx=0 tx=0).stop 2023-06-23T18:35:59.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7f3db597b700 1 -- 172.21.15.79:0/126849275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3db015c4b0 msgr2=0x7f3db01482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7f3db597b700 1 --2- 172.21.15.79:0/126849275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3db015c4b0 0x7f3db01482f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3da000c750 tx=0x7f3da000cb10 comp rx=0 tx=0).stop 2023-06-23T18:35:59.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7f3daf7fe700 1 -- 172.21.15.79:0/126849275 reap_dead start 2023-06-23T18:35:59.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.369+0000 7fd7e37fe700 1 --2- 172.21.15.79:0/2658829691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e40c52f0 0x7fd7e40aed10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800c500 tx=0x7fd7d800c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7fd739103700 1 -- 172.21.15.79:0/655406181 shutdown_connections 2023-06-23T18:35:59.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7fd739103700 1 -- 172.21.15.79:0/655406181 >> 172.21.15.79:0/655406181 conn(0x7fd7340cb1c0 msgr2=0x7fd7340cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7fd739103700 1 -- 172.21.15.79:0/655406181 shutdown_connections 2023-06-23T18:35:59.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7fd7e0ff9700 1 -- 172.21.15.79:0/2658829691 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8005d70 con 0x7fd7e40c52f0 2023-06-23T18:35:59.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7f3db597b700 1 -- 172.21.15.79:0/126849275 shutdown_connections 2023-06-23T18:35:59.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7f3db597b700 1 -- 172.21.15.79:0/126849275 >> 172.21.15.79:0/126849275 conn(0x7f3db00cb1c0 msgr2=0x7f3db0070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7fd7e0ff9700 1 -- 172.21.15.79:0/2658829691 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d80070d0 con 0x7fd7e40c52f0 2023-06-23T18:35:59.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7fd739103700 1 -- 172.21.15.79:0/655406181 wait complete. 2023-06-23T18:35:59.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7f3db597b700 1 -- 172.21.15.79:0/126849275 shutdown_connections 2023-06-23T18:35:59.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7f3db597b700 1 -- 172.21.15.79:0/126849275 wait complete. 2023-06-23T18:35:59.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7fd7ea53d700 1 -- 172.21.15.79:0/2658829691 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e4150180 con 0x7fd7e40c52f0 2023-06-23T18:35:59.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7fd7ea53d700 1 -- 172.21.15.79:0/2658829691 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e4150740 con 0x7fd7e40c52f0 2023-06-23T18:35:59.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.370+0000 7fd7e0ff9700 1 -- 172.21.15.79:0/2658829691 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8007440 con 0x7fd7e40c52f0 2023-06-23T18:35:59.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.371+0000 7fd7ea53d700 1 -- 172.21.15.79:0/2658829691 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd7c8000fc0 con 0x7fd7e40c52f0 2023-06-23T18:35:59.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.373+0000 7f5cada2b700 1 -- 172.21.15.79:0/3337142661 --> [v2:172.21.15.79:6800/2920568506,v1:172.21.15.79:6801/2920568506] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5ca8165130 con 0x7f5c94073490 2023-06-23T18:35:59.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.373+0000 7fd7e0ff9700 1 -- 172.21.15.79:0/2658829691 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd7d80075a0 con 0x7fd7e40c52f0 2023-06-23T18:35:59.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.373+0000 7fd7e0ff9700 1 --2- 172.21.15.79:0/2658829691 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd7cc06adf0 0x7fd7cc06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.373+0000 7fd7e0ff9700 1 -- 172.21.15.79:0/2658829691 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd7d8088580 con 0x7fd7e40c52f0 2023-06-23T18:35:59.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.373+0000 7fd7e3fff700 1 --2- 172.21.15.79:0/2658829691 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd7cc06adf0 0x7fd7cc06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.373+0000 7fd7e0ff9700 1 --2- 172.21.15.79:0/2658829691 >> [v2:172.21.15.79:6824/1549244775,v1:172.21.15.79:6825/1549244775] conn(0x7fd7cc073620 0x7fd7cc075a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.373+0000 7fd7e0ff9700 1 -- 172.21.15.79:0/2658829691 --> [v2:172.21.15.79:6824/1549244775,v1:172.21.15.79:6825/1549244775] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd7cc0760e0 con 0x7fd7cc073620 2023-06-23T18:35:59.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.373+0000 7fd7e0ff9700 1 -- 172.21.15.79:0/2658829691 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7d808ac10 con 0x7fd7e40c52f0 2023-06-23T18:35:59.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.374+0000 7f5c8ffff700 1 -- 172.21.15.79:0/3337142661 <== osd.1 v2:172.21.15.79:6800/2920568506 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5ca00092e0 con 0x7f5c94073490 2023-06-23T18:35:59.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.374+0000 7fd7e8ada700 1 --2- 172.21.15.79:0/2658829691 >> [v2:172.21.15.79:6824/1549244775,v1:172.21.15.79:6825/1549244775] conn(0x7fd7cc073620 0x7fd7cc075a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.374+0000 7fd7e3fff700 1 --2- 172.21.15.79:0/2658829691 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd7cc06adf0 0x7fd7cc06d2a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4006fd0 tx=0x7fd7d4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.374+0000 7fd7e8ada700 1 --2- 172.21.15.79:0/2658829691 >> [v2:172.21.15.79:6824/1549244775,v1:172.21.15.79:6825/1549244775] conn(0x7fd7cc073620 0x7fd7cc075a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.376+0000 7f5cada2b700 1 -- 172.21.15.79:0/3337142661 >> [v2:172.21.15.79:6800/2920568506,v1:172.21.15.79:6801/2920568506] conn(0x7f5c94073490 msgr2=0x7f5c940758a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.376+0000 7f5cada2b700 1 --2- 172.21.15.79:0/3337142661 >> [v2:172.21.15.79:6800/2920568506,v1:172.21.15.79:6801/2920568506] conn(0x7f5c94073490 0x7f5c940758a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.376+0000 7f5cada2b700 1 -- 172.21.15.79:0/3337142661 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5c9406ace0 msgr2=0x7f5c9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.376+0000 7f5cada2b700 1 --2- 172.21.15.79:0/3337142661 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5c9406ace0 0x7f5c9406d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8154c00 tx=0x7f5c98009040 comp rx=0 tx=0).stop 2023-06-23T18:35:59.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.376+0000 7f5cada2b700 1 -- 172.21.15.79:0/3337142661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca80c4dd0 msgr2=0x7f5ca80b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.376+0000 7f5cada2b700 1 --2- 172.21.15.79:0/3337142661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca80c4dd0 0x7f5ca80b1b30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00b530 tx=0x7f5c9c00b8f0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.376+0000 7f5ca77fe700 1 -- 172.21.15.79:0/3337142661 reap_dead start 2023-06-23T18:35:59.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.376+0000 7f5cada2b700 1 -- 172.21.15.79:0/3337142661 shutdown_connections 2023-06-23T18:35:59.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.376+0000 7f5cada2b700 1 -- 172.21.15.79:0/3337142661 >> 172.21.15.79:0/3337142661 conn(0x7f5ca80cb1c0 msgr2=0x7f5ca8069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.376+0000 7fd7e0ff9700 1 -- 172.21.15.79:0/2658829691 <== osd.3 v2:172.21.15.79:6824/1549244775 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd7cc0760e0 con 0x7fd7cc073620 2023-06-23T18:35:59.471 INFO:teuthology.orchestra.run.smithi079.stdout:55834574854 2023-06-23T18:35:59.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-06-23T18:35:59.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.377+0000 7f5cada2b700 1 -- 172.21.15.79:0/3337142661 shutdown_connections 2023-06-23T18:35:59.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.377+0000 7f5cada2b700 1 -- 172.21.15.79:0/3337142661 wait complete. 2023-06-23T18:35:59.473 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-23T18:35:59.474 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-06-23T18:35:59.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.382+0000 7f427b574700 1 -- 172.21.15.79:0/1130067924 --> [v2:172.21.15.155:6816/2160815572,v1:172.21.15.155:6817/2160815572] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f42740cf2b0 con 0x7f427415abb0 2023-06-23T18:35:59.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.383+0000 7f4269ffb700 1 -- 172.21.15.79:0/1130067924 <== osd.4 v2:172.21.15.155:6816/2160815572 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f426c00b8d0 con 0x7f427415abb0 2023-06-23T18:35:59.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.383+0000 7f159343e700 1 -- 172.21.15.79:0/2690148771 --> [v2:172.21.15.79:6816/1735841648,v1:172.21.15.79:6817/1735841648] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f158c155f90 con 0x7f1570073510 2023-06-23T18:35:59.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.387+0000 7f427b574700 1 -- 172.21.15.79:0/1130067924 >> [v2:172.21.15.155:6816/2160815572,v1:172.21.15.155:6817/2160815572] conn(0x7f427415abb0 msgr2=0x7f4274153ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.387+0000 7f427b574700 1 --2- 172.21.15.79:0/1130067924 >> [v2:172.21.15.155:6816/2160815572,v1:172.21.15.155:6817/2160815572] conn(0x7f427415abb0 0x7f4274153ac0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.387+0000 7f1581ffb700 1 -- 172.21.15.79:0/2690148771 <== osd.2 v2:172.21.15.79:6816/1735841648 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1588009280 con 0x7f1570073510 2023-06-23T18:35:59.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.388+0000 7f159343e700 1 -- 172.21.15.79:0/2690148771 >> [v2:172.21.15.79:6816/1735841648,v1:172.21.15.79:6817/1735841648] conn(0x7f1570073510 msgr2=0x7f1570075920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.388+0000 7f159343e700 1 --2- 172.21.15.79:0/2690148771 >> [v2:172.21.15.79:6816/1735841648,v1:172.21.15.79:6817/1735841648] conn(0x7f1570073510 0x7f1570075920 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.388+0000 7f159343e700 1 -- 172.21.15.79:0/2690148771 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f157006ace0 msgr2=0x7f157006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.388+0000 7f159343e700 1 --2- 172.21.15.79:0/2690148771 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f157006ace0 0x7f157006d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f157c009c10 tx=0x7f157c008030 comp rx=0 tx=0).stop 2023-06-23T18:35:59.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.388+0000 7f159343e700 1 -- 172.21.15.79:0/2690148771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f158c0c90c0 msgr2=0x7f158c0c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.388+0000 7f159343e700 1 --2- 172.21.15.79:0/2690148771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f158c0c90c0 0x7f158c0c6910 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f158400cbd0 tx=0x7f158400cf90 comp rx=0 tx=0).stop 2023-06-23T18:35:59.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.389+0000 7f15919db700 1 -- 172.21.15.79:0/2690148771 reap_dead start 2023-06-23T18:35:59.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.389+0000 7f159343e700 1 -- 172.21.15.79:0/2690148771 shutdown_connections 2023-06-23T18:35:59.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.389+0000 7f159343e700 1 -- 172.21.15.79:0/2690148771 >> 172.21.15.79:0/2690148771 conn(0x7f158c06b350 msgr2=0x7f158c160960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.389+0000 7f427b574700 1 -- 172.21.15.79:0/1130067924 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f425806ad30 msgr2=0x7f425806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.389+0000 7f427b574700 1 --2- 172.21.15.79:0/1130067924 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f425806ad30 0x7f425806d1e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f42740c9d50 tx=0x7f42640090e0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.389+0000 7f427b574700 1 -- 172.21.15.79:0/1130067924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4274160590 msgr2=0x7f42740b7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.389+0000 7f427b574700 1 --2- 172.21.15.79:0/1130067924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4274160590 0x7f42740b7bc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f427000ebe0 tx=0x7f427000eef0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.389+0000 7f4279b11700 1 -- 172.21.15.79:0/1130067924 reap_dead start 2023-06-23T18:35:59.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.389+0000 7f159343e700 1 -- 172.21.15.79:0/2690148771 shutdown_connections 2023-06-23T18:35:59.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.389+0000 7f159343e700 1 -- 172.21.15.79:0/2690148771 wait complete. 2023-06-23T18:35:59.482 INFO:teuthology.orchestra.run.smithi079.stdout:55834574854 2023-06-23T18:35:59.483 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-06-23T18:35:59.484 INFO:teuthology.orchestra.run.smithi079.stdout:55834574854 2023-06-23T18:35:59.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-06-23T18:35:59.486 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-23T18:35:59.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-06-23T18:35:59.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.395+0000 7f427b574700 1 -- 172.21.15.79:0/1130067924 shutdown_connections 2023-06-23T18:35:59.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.395+0000 7f427b574700 1 -- 172.21.15.79:0/1130067924 >> 172.21.15.79:0/1130067924 conn(0x7f427406b350 msgr2=0x7f4274160960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.395+0000 7f427b574700 1 -- 172.21.15.79:0/1130067924 shutdown_connections 2023-06-23T18:35:59.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.395+0000 7f427b574700 1 -- 172.21.15.79:0/1130067924 wait complete. 2023-06-23T18:35:59.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.401+0000 7fd7ea53d700 1 -- 172.21.15.79:0/2658829691 --> [v2:172.21.15.79:6824/1549244775,v1:172.21.15.79:6825/1549244775] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd7c0000f80 con 0x7fd7cc073620 2023-06-23T18:35:59.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.401+0000 7fd7e0ff9700 1 -- 172.21.15.79:0/2658829691 <== osd.3 v2:172.21.15.79:6824/1549244775 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd7c0000f80 con 0x7fd7cc073620 2023-06-23T18:35:59.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.401+0000 7fd7ea53d700 1 -- 172.21.15.79:0/2658829691 >> [v2:172.21.15.79:6824/1549244775,v1:172.21.15.79:6825/1549244775] conn(0x7fd7cc073620 msgr2=0x7fd7cc075a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.402+0000 7fd7ea53d700 1 --2- 172.21.15.79:0/2658829691 >> [v2:172.21.15.79:6824/1549244775,v1:172.21.15.79:6825/1549244775] conn(0x7fd7cc073620 0x7fd7cc075a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.402+0000 7fd7ea53d700 1 -- 172.21.15.79:0/2658829691 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd7cc06adf0 msgr2=0x7fd7cc06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.402+0000 7fd7ea53d700 1 --2- 172.21.15.79:0/2658829691 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd7cc06adf0 0x7fd7cc06d2a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4006fd0 tx=0x7fd7d4009190 comp rx=0 tx=0).stop 2023-06-23T18:35:59.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.402+0000 7fd7ea53d700 1 -- 172.21.15.79:0/2658829691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e40c52f0 msgr2=0x7fd7e40aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.402+0000 7fd7ea53d700 1 --2- 172.21.15.79:0/2658829691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e40c52f0 0x7fd7e40aed10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800c500 tx=0x7fd7d800c810 comp rx=0 tx=0).stop 2023-06-23T18:35:59.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.402+0000 7fd7e8ada700 1 -- 172.21.15.79:0/2658829691 reap_dead start 2023-06-23T18:35:59.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.402+0000 7fd7ea53d700 1 -- 172.21.15.79:0/2658829691 shutdown_connections 2023-06-23T18:35:59.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.402+0000 7fd7ea53d700 1 -- 172.21.15.79:0/2658829691 >> 172.21.15.79:0/2658829691 conn(0x7fd7e40cb1c0 msgr2=0x7fd7e414f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.402+0000 7fd7ea53d700 1 -- 172.21.15.79:0/2658829691 shutdown_connections 2023-06-23T18:35:59.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.402+0000 7fd7ea53d700 1 -- 172.21.15.79:0/2658829691 wait complete. 2023-06-23T18:35:59.493 INFO:teuthology.orchestra.run.smithi079.stdout:55834574853 2023-06-23T18:35:59.493 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-06-23T18:35:59.494 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-23T18:35:59.494 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-06-23T18:35:59.495 INFO:teuthology.orchestra.run.smithi079.stdout:55834574853 2023-06-23T18:35:59.495 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-06-23T18:35:59.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.717+0000 7fbd18079700 1 Processor -- start 2023-06-23T18:35:59.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.717+0000 7fbd18079700 1 -- start start 2023-06-23T18:35:59.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd18079700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd10071490 0x7fbd10071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd18079700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbd100b1270 0x7fbd10071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd18079700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbd100aee30 0x7fbd10069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd18079700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd101608f0 con 0x7fbd100aee30 2023-06-23T18:35:59.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd18079700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd100722e0 con 0x7fbd100b1270 2023-06-23T18:35:59.725 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd18079700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd10160e20 con 0x7fbd10071490 2023-06-23T18:35:59.725 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd16616700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbd100aee30 0x7fbd10069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.725 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd16616700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbd100aee30 0x7fbd10069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39350/0 (socket says 172.21.15.79:39350) 2023-06-23T18:35:59.725 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd16616700 1 -- 172.21.15.79:0/1444474822 learned_addr learned my addr 172.21.15.79:0/1444474822 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd15614700 1 --2- 172.21.15.79:0/1444474822 >> v2:172.21.15.155:3300/0 conn(0x7fbd100b1270 0x7fbd10071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd15614700 1 -- 172.21.15.79:0/1444474822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd10071490 msgr2=0x7fbd10071860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:35:59.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd15614700 1 --2- 172.21.15.79:0/1444474822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd10071490 0x7fbd10071860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.727 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd15614700 1 -- 172.21.15.79:0/1444474822 >> v2:172.21.15.79:3300/0 conn(0x7fbd100aee30 msgr2=0x7fbd10069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.727 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd15614700 1 --2- 172.21.15.79:0/1444474822 >> v2:172.21.15.79:3300/0 conn(0x7fbd100aee30 0x7fbd10069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.727 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.718+0000 7fbd15614700 1 -- 172.21.15.79:0/1444474822 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd100cd4f0 con 0x7fbd100b1270 2023-06-23T18:35:59.727 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.719+0000 7fbd15614700 1 --2- 172.21.15.79:0/1444474822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd100b1270 0x7fbd10071da0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbd100709b0 tx=0x7fbd0c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7442390c7e57a29c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.728 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.719+0000 7f3c38254700 1 Processor -- start 2023-06-23T18:35:59.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.719+0000 7fbd14e13700 1 -- 172.21.15.79:0/1444474822 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c011830 con 0x7fbd100b1270 2023-06-23T18:35:59.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.719+0000 7fbd14e13700 1 -- 172.21.15.79:0/1444474822 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c011990 con 0x7fbd100b1270 2023-06-23T18:35:59.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.719+0000 7fbd14e13700 1 -- 172.21.15.79:0/1444474822 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c011d20 con 0x7fbd100b1270 2023-06-23T18:35:59.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.719+0000 7f8dbbd44700 1 Processor -- start 2023-06-23T18:35:59.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.719+0000 7f3c38254700 1 -- start start 2023-06-23T18:35:59.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.720+0000 7f3c38254700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c30160590 0x7f3c300d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.720+0000 7f3c38254700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3c30164e90 0x7f3c300cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.720+0000 7f3c38254700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3c300cfd10 0x7f3c30164280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.720+0000 7f3c38254700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c300ce730 con 0x7f3c300cfd10 2023-06-23T18:35:59.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.720+0000 7f3c38254700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c300cea10 con 0x7f3c30164e90 2023-06-23T18:35:59.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.720+0000 7f3c38254700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300cecf0 con 0x7f3c30160590 2023-06-23T18:35:59.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.720+0000 7f8dbbd44700 1 -- start start 2023-06-23T18:35:59.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.720+0000 7fbd18079700 1 -- 172.21.15.79:0/1444474822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd100b1270 msgr2=0x7fbd10071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.720+0000 7fbd18079700 1 --2- 172.21.15.79:0/1444474822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd100b1270 0x7fbd10071da0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbd100709b0 tx=0x7fbd0c009b20 comp rx=0 tx=0).stop 2023-06-23T18:35:59.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.720+0000 7fbd18079700 1 -- 172.21.15.79:0/1444474822 shutdown_connections 2023-06-23T18:35:59.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.720+0000 7fbd18079700 1 --2- 172.21.15.79:0/1444474822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd10071490 0x7fbd10071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.720+0000 7fbd18079700 1 --2- 172.21.15.79:0/1444474822 >> v2:172.21.15.79:3300/0 conn(0x7fbd100aee30 0x7fbd10069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.720+0000 7fbd18079700 1 --2- 172.21.15.79:0/1444474822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd100b1270 0x7fbd10071da0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.720+0000 7fbd18079700 1 -- 172.21.15.79:0/1444474822 >> 172.21.15.79:0/1444474822 conn(0x7fbd1006d660 msgr2=0x7fbd1006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.721+0000 7f3c367f1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3c300cfd10 0x7f3c30164280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.721+0000 7f3c367f1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3c300cfd10 0x7f3c30164280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39358/0 (socket says 172.21.15.79:39358) 2023-06-23T18:35:59.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.721+0000 7f3c367f1700 1 -- 172.21.15.79:0/3087554516 learned_addr learned my addr 172.21.15.79:0/3087554516 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.721+0000 7f8dbbd44700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8db406b0b0 0x7f8db406df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.721+0000 7f8dbbd44700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db40c1dd0 con 0x7f8db40cad80 2023-06-23T18:35:59.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.721+0000 7f8dbbd44700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db40cb150 con 0x7f8db4070b10 2023-06-23T18:35:59.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.721+0000 7f8dbbd44700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db40c2630 con 0x7f8db406b0b0 2023-06-23T18:35:59.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.721+0000 7f8dba2e1700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f8db4070b10 0x7f8db41645c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:50670/0 (socket says 172.21.15.79:50670) 2023-06-23T18:35:59.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.721+0000 7f8dba2e1700 1 -- 172.21.15.79:0/2810498918 learned_addr learned my addr 172.21.15.79:0/2810498918 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:59.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.721+0000 7f3c367f1700 1 -- 172.21.15.79:0/3087554516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c30160590 msgr2=0x7f3c300d10f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:35:59.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.721+0000 7f3c367f1700 1 --2- 172.21.15.79:0/3087554516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c30160590 0x7f3c300d10f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.721+0000 7f3c367f1700 1 -- 172.21.15.79:0/3087554516 >> v2:172.21.15.155:3300/0 conn(0x7f3c30164e90 msgr2=0x7f3c300cf7d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:35:59.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.721+0000 7f3c367f1700 1 --2- 172.21.15.79:0/3087554516 >> v2:172.21.15.155:3300/0 conn(0x7f3c30164e90 0x7f3c300cf7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.722+0000 7f8db92df700 1 --2- 172.21.15.79:0/2810498918 >> v2:172.21.15.79:3300/0 conn(0x7f8db406b0b0 0x7f8db406df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.722+0000 7fbd18079700 1 -- 172.21.15.79:0/1444474822 shutdown_connections 2023-06-23T18:35:59.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.722+0000 7f3c367f1700 1 -- 172.21.15.79:0/3087554516 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c300cefd0 con 0x7f3c300cfd10 2023-06-23T18:35:59.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.722+0000 7fbd18079700 1 -- 172.21.15.79:0/1444474822 wait complete. 2023-06-23T18:35:59.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.722+0000 7f8db8ade700 1 -- 172.21.15.79:0/2810498918 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3539011060 0 0) 0x7f8db40cb150 con 0x7f8db4070b10 2023-06-23T18:35:59.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.722+0000 7f8db8ade700 1 -- 172.21.15.79:0/2810498918 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8da00035b0 con 0x7f8db4070b10 2023-06-23T18:35:59.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.722+0000 7f3c367f1700 1 --2- 172.21.15.79:0/3087554516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c300cfd10 0x7f3c30164280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3c300d06c0 tx=0x7f3c2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d742a129d1be816f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.722+0000 7f8db92df700 1 -- 172.21.15.79:0/2810498918 >> v1:172.21.15.155:6789/0 conn(0x7f8db4070b10 legacy=0x7f8db41645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.722+0000 7f8db92df700 1 -- 172.21.15.79:0/2810498918 >> v1:172.21.15.79:6789/0 conn(0x7f8db40cad80 legacy=0x7f8db40cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.722+0000 7f8db92df700 1 -- 172.21.15.79:0/2810498918 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db40c2070 con 0x7f8db406b0b0 2023-06-23T18:35:59.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.722+0000 7f3c34fee700 1 -- 172.21.15.79:0/3087554516 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800e070 con 0x7f3c300cfd10 2023-06-23T18:35:59.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.722+0000 7f3c34fee700 1 -- 172.21.15.79:0/3087554516 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2800b9c0 con 0x7f3c300cfd10 2023-06-23T18:35:59.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.722+0000 7f3c34fee700 1 -- 172.21.15.79:0/3087554516 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800bd70 con 0x7f3c300cfd10 2023-06-23T18:35:59.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.722+0000 7f8db92df700 1 --2- 172.21.15.79:0/2810498918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8db406b0b0 0x7f8db406df40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8da40097f0 tx=0x7f8da4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4be8fc3c2cf954 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.723+0000 7fd5f71f2700 1 Processor -- start 2023-06-23T18:35:59.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.723+0000 7f8db8ade700 1 -- 172.21.15.79:0/2810498918 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0001670 con 0x7f8db406b0b0 2023-06-23T18:35:59.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.723+0000 7f8db8ade700 1 -- 172.21.15.79:0/2810498918 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da4004030 con 0x7f8db406b0b0 2023-06-23T18:35:59.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.723+0000 7f8db8ade700 1 -- 172.21.15.79:0/2810498918 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da40043c0 con 0x7f8db406b0b0 2023-06-23T18:35:59.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.723+0000 7fd5f71f2700 1 -- start start 2023-06-23T18:35:59.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.723+0000 7f3c38254700 1 -- 172.21.15.79:0/3087554516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c300cfd10 msgr2=0x7f3c30164280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.723+0000 7f3c38254700 1 --2- 172.21.15.79:0/3087554516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c300cfd10 0x7f3c30164280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3c300d06c0 tx=0x7f3c2800b330 comp rx=0 tx=0).stop 2023-06-23T18:35:59.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.723+0000 7f3c38254700 1 -- 172.21.15.79:0/3087554516 shutdown_connections 2023-06-23T18:35:59.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.723+0000 7f3c38254700 1 --2- 172.21.15.79:0/3087554516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c30160590 0x7f3c300d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.723+0000 7f3c38254700 1 --2- 172.21.15.79:0/3087554516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c300cfd10 0x7f3c30164280 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.723+0000 7f3c38254700 1 --2- 172.21.15.79:0/3087554516 >> v2:172.21.15.155:3300/0 conn(0x7f3c30164e90 0x7f3c300cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.723+0000 7f3c38254700 1 -- 172.21.15.79:0/3087554516 >> 172.21.15.79:0/3087554516 conn(0x7f3c3006b350 msgr2=0x7f3c3006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7f3c38254700 1 -- 172.21.15.79:0/3087554516 shutdown_connections 2023-06-23T18:35:59.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.723+0000 7f4c8d3c6700 1 Processor -- start 2023-06-23T18:35:59.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7f4c8d3c6700 1 -- start start 2023-06-23T18:35:59.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7fd5f71f2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd5f00adce0 0x7fd5f0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7fd5f71f2700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd5f01591a0 0x7fd5f006a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7fd5f71f2700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f0159a60 con 0x7fd5f015ccc0 2023-06-23T18:35:59.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7fd5f71f2700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f0159780 con 0x7fd5f00adce0 2023-06-23T18:35:59.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7fd5f71f2700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f015c1e0 con 0x7fd5f01591a0 2023-06-23T18:35:59.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7f8dbbd44700 1 -- 172.21.15.79:0/2810498918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8db406b0b0 msgr2=0x7f8db406df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7f8dbbd44700 1 --2- 172.21.15.79:0/2810498918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8db406b0b0 0x7f8db406df40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8da40097f0 tx=0x7f8da4009b00 comp rx=0 tx=0).stop 2023-06-23T18:35:59.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.723+0000 7fbd18079700 1 Processor -- start 2023-06-23T18:35:59.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7fd5f4f8e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd5f00adce0 0x7fd5f0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7fd5f4f8e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd5f00adce0 0x7fd5f0069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39366/0 (socket says 172.21.15.79:39366) 2023-06-23T18:35:59.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7fd5f4f8e700 1 -- 172.21.15.79:0/199864463 learned_addr learned my addr 172.21.15.79:0/199864463 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7fd5f4f8e700 1 -- 172.21.15.79:0/199864463 >> v2:172.21.15.155:3300/0 conn(0x7fd5f01591a0 msgr2=0x7fd5f006a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7fd5f4f8e700 1 --2- 172.21.15.79:0/199864463 >> v2:172.21.15.155:3300/0 conn(0x7fd5f01591a0 0x7fd5f006a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7fd5f4f8e700 1 -- 172.21.15.79:0/199864463 >> v1:172.21.15.155:6789/0 conn(0x7fd5f015ccc0 legacy=0x7fd5f0158a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:35:59.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7fd5f4f8e700 1 -- 172.21.15.79:0/199864463 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f015c4c0 con 0x7fd5f00adce0 2023-06-23T18:35:59.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7f3c38254700 1 -- 172.21.15.79:0/3087554516 wait complete. 2023-06-23T18:35:59.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7fd5f4f8e700 1 --2- 172.21.15.79:0/199864463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f00adce0 0x7fd5f0069e90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40088f0 tx=0x7fd5e400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d94beec0ab492b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7fbd18079700 1 -- start start 2023-06-23T18:35:59.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7fbd18079700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd10071490 0x7fbd100c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fbd18079700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aee30 0x7fbd1015eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fbd18079700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd100b1270 0x7fbd1015f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fbd18079700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd100c5af0 con 0x7fbd100aee30 2023-06-23T18:35:59.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fbd18079700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd100c5250 con 0x7fbd100b1270 2023-06-23T18:35:59.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fbd18079700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd100c5530 con 0x7fbd10071490 2023-06-23T18:35:59.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7f3c38254700 1 Processor -- start 2023-06-23T18:35:59.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7f8dbbd44700 1 -- 172.21.15.79:0/2810498918 shutdown_connections 2023-06-23T18:35:59.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7f8dbbd44700 1 --2- 172.21.15.79:0/2810498918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8db406b0b0 0x7f8db406df40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.724+0000 7f8dbbd44700 1 -- 172.21.15.79:0/2810498918 >> 172.21.15.79:0/2810498918 conn(0x7f8db40b17e0 msgr2=0x7f8db406cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7f8dbbd44700 1 -- 172.21.15.79:0/2810498918 shutdown_connections 2023-06-23T18:35:59.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7f8dbbd44700 1 -- 172.21.15.79:0/2810498918 wait complete. 2023-06-23T18:35:59.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fbd15614700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aee30 0x7fbd1015eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fbd15614700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aee30 0x7fbd1015eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39372/0 (socket says 172.21.15.79:39372) 2023-06-23T18:35:59.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fbd15614700 1 -- 172.21.15.79:0/3128999824 learned_addr learned my addr 172.21.15.79:0/3128999824 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7f4c8d3c6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c881608e0 0x7f4c88069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7f4c8d3c6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4c880c52f0 0x7f4c8806a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7f4c8d3c6700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c88164890 con 0x7f4c8806e5f0 2023-06-23T18:35:59.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7f4c8d3c6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c880c91c0 con 0x7f4c880c52f0 2023-06-23T18:35:59.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7f4c8d3c6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c88164b70 con 0x7f4c881608e0 2023-06-23T18:35:59.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7f4c877fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f4c8806e5f0 0x7f4c88070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42412/0 (socket says 172.21.15.79:42412) 2023-06-23T18:35:59.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.725+0000 7f4c877fe700 1 -- 172.21.15.79:0/4132146377 learned_addr learned my addr 172.21.15.79:0/4132146377 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:59.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7f4c867fc700 1 --2- 172.21.15.79:0/4132146377 >> v2:172.21.15.79:3300/0 conn(0x7f4c880c52f0 0x7f4c8806a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7f4c85ffb700 1 -- 172.21.15.79:0/4132146377 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455843343 0 0) 0x7f4c88164890 con 0x7f4c8806e5f0 2023-06-23T18:35:59.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7f4c85ffb700 1 -- 172.21.15.79:0/4132146377 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c700035b0 con 0x7f4c8806e5f0 2023-06-23T18:35:59.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7f3c38254700 1 -- start start 2023-06-23T18:35:59.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fbd15614700 1 -- 172.21.15.79:0/3128999824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd10071490 msgr2=0x7fbd100c49f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T18:35:59.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fbd15614700 1 --2- 172.21.15.79:0/3128999824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd10071490 0x7fbd100c49f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fbd15614700 1 -- 172.21.15.79:0/3128999824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd100b1270 msgr2=0x7fbd1015f410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-23T18:35:59.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fbd15614700 1 --2- 172.21.15.79:0/3128999824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd100b1270 0x7fbd1015f410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fbd15614700 1 -- 172.21.15.79:0/3128999824 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c0094a0 con 0x7fbd100aee30 2023-06-23T18:35:59.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fd5ef7fe700 1 -- 172.21.15.79:0/199864463 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4010070 con 0x7fd5f00adce0 2023-06-23T18:35:59.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fd5ef7fe700 1 -- 172.21.15.79:0/199864463 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e400da00 con 0x7fd5f00adce0 2023-06-23T18:35:59.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7f4c86ffd700 1 --2- 172.21.15.79:0/4132146377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c881608e0 0x7f4c88069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7f4c867fc700 1 -- 172.21.15.79:0/4132146377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c881608e0 msgr2=0x7f4c88069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7fd5ef7fe700 1 -- 172.21.15.79:0/199864463 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400dd70 con 0x7fd5f00adce0 2023-06-23T18:35:59.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7f8dbbd44700 1 Processor -- start 2023-06-23T18:35:59.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7f4c867fc700 1 --2- 172.21.15.79:0/4132146377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c881608e0 0x7f4c88069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7f4c867fc700 1 -- 172.21.15.79:0/4132146377 >> v1:172.21.15.79:6789/0 conn(0x7f4c8806e5f0 legacy=0x7f4c88070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7f4c867fc700 1 -- 172.21.15.79:0/4132146377 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c88164e50 con 0x7f4c880c52f0 2023-06-23T18:35:59.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.727+0000 7fbd15614700 1 --2- 172.21.15.79:0/3128999824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aee30 0x7fbd1015eed0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c014040 tx=0x7fbd0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.727+0000 7fbd067fc700 1 -- 172.21.15.79:0/3128999824 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0036a0 con 0x7fbd100aee30 2023-06-23T18:35:59.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.727+0000 7fbd067fc700 1 -- 172.21.15.79:0/3128999824 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c01e070 con 0x7fbd100aee30 2023-06-23T18:35:59.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.727+0000 7fbd067fc700 1 -- 172.21.15.79:0/3128999824 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c003a40 con 0x7fbd100aee30 2023-06-23T18:35:59.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.727+0000 7fbd18079700 1 -- 172.21.15.79:0/3128999824 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd100c5810 con 0x7fbd100aee30 2023-06-23T18:35:59.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.727+0000 7fbd18079700 1 -- 172.21.15.79:0/3128999824 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd100c4f30 con 0x7fbd100aee30 2023-06-23T18:35:59.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.727+0000 7fd5f71f2700 1 -- 172.21.15.79:0/199864463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f00adce0 msgr2=0x7fd5f0069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.727+0000 7fd5f71f2700 1 --2- 172.21.15.79:0/199864463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f00adce0 0x7fd5f0069e90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40088f0 tx=0x7fd5e400d330 comp rx=0 tx=0).stop 2023-06-23T18:35:59.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.727+0000 7fd5f71f2700 1 -- 172.21.15.79:0/199864463 shutdown_connections 2023-06-23T18:35:59.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.727+0000 7fd5f71f2700 1 --2- 172.21.15.79:0/199864463 >> v2:172.21.15.155:3300/0 conn(0x7fd5f01591a0 0x7fd5f006a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.727+0000 7fd5f71f2700 1 --2- 172.21.15.79:0/199864463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f00adce0 0x7fd5f0069e90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.727+0000 7fd5f71f2700 1 -- 172.21.15.79:0/199864463 >> 172.21.15.79:0/199864463 conn(0x7fd5f00cb1c0 msgr2=0x7fd5f00cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.726+0000 7f8dbbd44700 1 -- start start 2023-06-23T18:35:59.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.728+0000 7f8dbbd44700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8db406b0b0 0x7f8db40b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.728+0000 7f8dbbd44700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8db4070b10 0x7f8db40c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.728+0000 7f8dbbd44700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8db40cad80 0x7f8db414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.728+0000 7f8dbbd44700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40b45e0 con 0x7f8db40cad80 2023-06-23T18:35:59.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.728+0000 7f8dbbd44700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40c92f0 con 0x7f8db4070b10 2023-06-23T18:35:59.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.728+0000 7f8dbbd44700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40c62f0 con 0x7f8db406b0b0 2023-06-23T18:35:59.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.728+0000 7fbcba2d1700 1 Processor -- start 2023-06-23T18:35:59.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.728+0000 7fbcba2d1700 1 -- start start 2023-06-23T18:35:59.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.728+0000 7fd5f71f2700 1 -- 172.21.15.79:0/199864463 shutdown_connections 2023-06-23T18:35:59.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.728+0000 7fd5f71f2700 1 -- 172.21.15.79:0/199864463 wait complete. 2023-06-23T18:35:59.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.728+0000 7f4c867fc700 1 --2- 172.21.15.79:0/4132146377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c880c52f0 0x7f4c8806a3d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009840 tx=0x7f4c78009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a2fa7e9d4c20 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.728+0000 7f8dba2e1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8db40cad80 0x7f8db414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.728+0000 7f8dba2e1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8db40cad80 0x7f8db414d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39384/0 (socket says 172.21.15.79:39384) 2023-06-23T18:35:59.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.728+0000 7f8dba2e1700 1 -- 172.21.15.79:0/852375791 learned_addr learned my addr 172.21.15.79:0/852375791 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7fbcba2d1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbcb40c91c0 0x7fbcb4070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7fbcba2d1700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb4069e90 0x7fbcb406a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7fbcba2d1700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb40ae160 con 0x7fbcb40add90 2023-06-23T18:35:59.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7fbcba2d1700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb40c9660 con 0x7fbcb40c91c0 2023-06-23T18:35:59.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7fbcba2d1700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb406a850 con 0x7fbcb4069e90 2023-06-23T18:35:59.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7f8dba2e1700 1 -- 172.21.15.79:0/852375791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8db406b0b0 msgr2=0x7f8db40b4a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-23T18:35:59.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7f8dba2e1700 1 --2- 172.21.15.79:0/852375791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8db406b0b0 0x7f8db40b4a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7f8dba2e1700 1 -- 172.21.15.79:0/852375791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8db4070b10 msgr2=0x7f8db40c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7f8dba2e1700 1 --2- 172.21.15.79:0/852375791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8db4070b10 0x7f8db40c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7f8dba2e1700 1 -- 172.21.15.79:0/852375791 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da40094a0 con 0x7f8db40cad80 2023-06-23T18:35:59.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7f8dba2e1700 1 --2- 172.21.15.79:0/852375791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8db40cad80 0x7f8db414d3e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8db000c700 tx=0x7f8db000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7f8daa7fc700 1 -- 172.21.15.79:0/852375791 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0007560 con 0x7f8db40cad80 2023-06-23T18:35:59.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7f8daa7fc700 1 -- 172.21.15.79:0/852375791 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db00076e0 con 0x7f8db40cad80 2023-06-23T18:35:59.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7f8daa7fc700 1 -- 172.21.15.79:0/852375791 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0004ab0 con 0x7f8db40cad80 2023-06-23T18:35:59.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7f4c85ffb700 1 -- 172.21.15.79:0/4132146377 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78011830 con 0x7f4c880c52f0 2023-06-23T18:35:59.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7fbd18079700 1 -- 172.21.15.79:0/3128999824 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd101571b0 con 0x7fbd100aee30 2023-06-23T18:35:59.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7fbd067fc700 1 -- 172.21.15.79:0/3128999824 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbd0c003ba0 con 0x7fbd100aee30 2023-06-23T18:35:59.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7fbcb3fff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fbcb40add90 0x7fbcb40ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:50698/0 (socket says 172.21.15.79:50698) 2023-06-23T18:35:59.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7fbcb3fff700 1 -- 172.21.15.79:0/1494348560 learned_addr learned my addr 172.21.15.79:0/1494348560 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:35:59.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7fbcb886e700 1 --2- 172.21.15.79:0/1494348560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb4069e90 0x7fbcb406a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7fbcb37fe700 1 --2- 172.21.15.79:0/1494348560 >> v2:172.21.15.79:3300/0 conn(0x7fbcb40c91c0 0x7fbcb4070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7fbd067fc700 1 --2- 172.21.15.79:0/3128999824 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbcf406ace0 0x7fbcf406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7fbd067fc700 1 -- 172.21.15.79:0/3128999824 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbd0c08a470 con 0x7fbd100aee30 2023-06-23T18:35:59.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7fd5f71f2700 1 Processor -- start 2023-06-23T18:35:59.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7fbd15e15700 1 --2- 172.21.15.79:0/3128999824 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbcf406ace0 0x7fbcf406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7fbcb886e700 1 -- 172.21.15.79:0/1494348560 >> v2:172.21.15.79:3300/0 conn(0x7fbcb40c91c0 msgr2=0x7fbcb4070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7fbcb886e700 1 --2- 172.21.15.79:0/1494348560 >> v2:172.21.15.79:3300/0 conn(0x7fbcb40c91c0 0x7fbcb4070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7fbcb886e700 1 -- 172.21.15.79:0/1494348560 >> v1:172.21.15.155:6789/0 conn(0x7fbcb40add90 legacy=0x7fbcb40ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7fbcb886e700 1 -- 172.21.15.79:0/1494348560 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb4071180 con 0x7fbcb4069e90 2023-06-23T18:35:59.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7fbcb886e700 1 --2- 172.21.15.79:0/1494348560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb4069e90 0x7fbcb406a310 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009840 tx=0x7fbca4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9965e39b7aeec657 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7f4c85ffb700 1 -- 172.21.15.79:0/4132146377 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c78011990 con 0x7f4c880c52f0 2023-06-23T18:35:59.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.729+0000 7f4c85ffb700 1 -- 172.21.15.79:0/4132146377 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78011d00 con 0x7f4c880c52f0 2023-06-23T18:35:59.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/4132146377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c880c52f0 msgr2=0x7f4c8806a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7f4c8d3c6700 1 --2- 172.21.15.79:0/4132146377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c880c52f0 0x7f4c8806a3d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009840 tx=0x7f4c78009b50 comp rx=0 tx=0).stop 2023-06-23T18:35:59.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7f8dbbd44700 1 -- 172.21.15.79:0/852375791 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db40b3960 con 0x7f8db40cad80 2023-06-23T18:35:59.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.730+0000 7f8dbbd44700 1 -- 172.21.15.79:0/852375791 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db40b3f20 con 0x7f8db40cad80 2023-06-23T18:35:59.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7fbd15e15700 1 --2- 172.21.15.79:0/3128999824 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbcf406ace0 0x7fbcf406d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd00006fd0 tx=0x7fbd00006bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/4132146377 shutdown_connections 2023-06-23T18:35:59.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7f4c8d3c6700 1 --2- 172.21.15.79:0/4132146377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c881608e0 0x7f4c88069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7f4c8d3c6700 1 --2- 172.21.15.79:0/4132146377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c880c52f0 0x7f4c8806a3d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/4132146377 >> 172.21.15.79:0/4132146377 conn(0x7f4c880cb1c0 msgr2=0x7f4c880cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7f3c38254700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c30160590 0x7f3c300cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7f3c38254700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c30164e90 0x7f3c30201c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7f3c38254700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c300b8080 0x7f3c300c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7f3c38254700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300cb020 con 0x7f3c300b8080 2023-06-23T18:35:59.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7f3c38254700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300cc5a0 con 0x7f3c30160590 2023-06-23T18:35:59.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7f3c38254700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c30202140 con 0x7f3c30164e90 2023-06-23T18:35:59.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7f3c35ff0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c30160590 0x7f3c300cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7f3c35ff0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c30160590 0x7f3c300cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:45620/0 (socket says 172.21.15.79:45620) 2023-06-23T18:35:59.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7f3c35ff0700 1 -- 172.21.15.79:0/3713069423 learned_addr learned my addr 172.21.15.79:0/3713069423 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7f3c357ef700 1 --2- 172.21.15.79:0/3713069423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c30164e90 0x7f3c30201c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7f3c367f1700 1 --2- 172.21.15.79:0/3713069423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c300b8080 0x7f3c300c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7fd5f71f2700 1 -- start start 2023-06-23T18:35:59.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7fd5f71f2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5f00adce0 0x7fd5f00c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7fd5f71f2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f01591a0 0x7fd5f00c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.731+0000 7f8dbbd44700 1 -- 172.21.15.79:0/852375791 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db415e230 con 0x7f8db40cad80 2023-06-23T18:35:59.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/4132146377 shutdown_connections 2023-06-23T18:35:59.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7efe857cc700 1 Processor -- start 2023-06-23T18:35:59.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7fd5f71f2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5f015ccc0 0x7fd5f006dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7fd5f71f2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f006d980 con 0x7fd5f01591a0 2023-06-23T18:35:59.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7fd5f71f2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f006cd00 con 0x7fd5f00adce0 2023-06-23T18:35:59.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7fd5f71f2700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f006cfe0 con 0x7fd5f015ccc0 2023-06-23T18:35:59.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7f3c35ff0700 1 -- 172.21.15.79:0/3713069423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c30164e90 msgr2=0x7f3c30201c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7f3c35ff0700 1 --2- 172.21.15.79:0/3713069423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c30164e90 0x7f3c30201c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7f3c35ff0700 1 -- 172.21.15.79:0/3713069423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c300b8080 msgr2=0x7f3c300c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7f3c35ff0700 1 --2- 172.21.15.79:0/3713069423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c300b8080 0x7f3c300c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7f3c35ff0700 1 -- 172.21.15.79:0/3713069423 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c28009b30 con 0x7f3c30160590 2023-06-23T18:35:59.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7fd5f4f8e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5f00adce0 0x7fd5f00c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7fd5f4f8e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5f00adce0 0x7fd5f00c5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:45628/0 (socket says 172.21.15.79:45628) 2023-06-23T18:35:59.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7fd5f4f8e700 1 -- 172.21.15.79:0/2491090489 learned_addr learned my addr 172.21.15.79:0/2491090489 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7f3c35ff0700 1 --2- 172.21.15.79:0/3713069423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c30160590 0x7f3c300cca30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00b580 tx=0x7f3c2c00b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7efe857cc700 1 -- start start 2023-06-23T18:35:59.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7fd5effff700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f01591a0 0x7fd5f00c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7f3c267fc700 1 -- 172.21.15.79:0/3713069423 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c011830 con 0x7f3c30160590 2023-06-23T18:35:59.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7f3c267fc700 1 -- 172.21.15.79:0/3713069423 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c0119b0 con 0x7f3c30160590 2023-06-23T18:35:59.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7f3c38254700 1 -- 172.21.15.79:0/3713069423 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c300cb990 con 0x7f3c30160590 2023-06-23T18:35:59.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7f3c38254700 1 -- 172.21.15.79:0/3713069423 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c300cbf50 con 0x7f3c30160590 2023-06-23T18:35:59.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7efe857cc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe8006a5a0 0x7efe800c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7efe857cc700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efe800c52f0 0x7efe800c4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7efe857cc700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe80164890 con 0x7efe801608e0 2023-06-23T18:35:59.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7efe857cc700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe800c91c0 con 0x7efe800c52f0 2023-06-23T18:35:59.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7efe857cc700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe80164b70 con 0x7efe8006a5a0 2023-06-23T18:35:59.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7efe7f7fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efe800c52f0 0x7efe800c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7efe7f7fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efe800c52f0 0x7efe800c4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:45636/0 (socket says 172.21.15.79:45636) 2023-06-23T18:35:59.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7efe7f7fe700 1 -- 172.21.15.79:0/611144571 learned_addr learned my addr 172.21.15.79:0/611144571 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7efe7e7fc700 1 --2- 172.21.15.79:0/611144571 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe8006a5a0 0x7efe800c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.732+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/4132146377 wait complete. 2023-06-23T18:35:59.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7f4c8d3c6700 1 Processor -- start 2023-06-23T18:35:59.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7fd5f578f700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5f015ccc0 0x7fd5f006dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7fd5f4f8e700 1 -- 172.21.15.79:0/2491090489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5f015ccc0 msgr2=0x7fd5f006dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7fd5f4f8e700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5f015ccc0 0x7fd5f006dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7fd5f4f8e700 1 -- 172.21.15.79:0/2491090489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f01591a0 msgr2=0x7fd5f00c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7fd5f4f8e700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f01591a0 0x7fd5f00c6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7fd5f4f8e700 1 -- 172.21.15.79:0/2491090489 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e40085d0 con 0x7fd5f00adce0 2023-06-23T18:35:59.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7fd5f578f700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5f015ccc0 0x7fd5f006dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:35:59.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7fd5f4f8e700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5f00adce0 0x7fd5f00c5780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400d370 tx=0x7fd5e4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7fd5ed7fa700 1 -- 172.21.15.79:0/2491090489 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4003dd0 con 0x7fd5f00adce0 2023-06-23T18:35:59.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7fd5ed7fa700 1 -- 172.21.15.79:0/2491090489 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e4008980 con 0x7fd5f00adce0 2023-06-23T18:35:59.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.733+0000 7fd5ed7fa700 1 -- 172.21.15.79:0/2491090489 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4008cf0 con 0x7fd5f00adce0 2023-06-23T18:35:59.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7fd5f71f2700 1 -- 172.21.15.79:0/2491090489 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f006d2c0 con 0x7fd5f00adce0 2023-06-23T18:35:59.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7fd5f71f2700 1 -- 172.21.15.79:0/2491090489 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f01506e0 con 0x7fd5f00adce0 2023-06-23T18:35:59.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7f3c267fc700 1 -- 172.21.15.79:0/3713069423 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c011d40 con 0x7f3c30160590 2023-06-23T18:35:59.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.735+0000 7f3c38254700 1 -- 172.21.15.79:0/3713069423 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c300d2e50 con 0x7f3c30160590 2023-06-23T18:35:59.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.735+0000 7efe7e7fc700 1 -- 172.21.15.79:0/611144571 >> v2:172.21.15.155:3300/0 conn(0x7efe800c52f0 msgr2=0x7efe800c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.735+0000 7efe7e7fc700 1 --2- 172.21.15.79:0/611144571 >> v2:172.21.15.155:3300/0 conn(0x7efe800c52f0 0x7efe800c4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.735+0000 7efe7e7fc700 1 -- 172.21.15.79:0/611144571 >> v1:172.21.15.79:6789/0 conn(0x7efe801608e0 legacy=0x7efe80069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.735+0000 7efe7e7fc700 1 -- 172.21.15.79:0/611144571 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe80164e50 con 0x7efe8006a5a0 2023-06-23T18:35:59.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.735+0000 7fd5f71f2700 1 -- 172.21.15.79:0/2491090489 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f0150870 con 0x7fd5f00adce0 2023-06-23T18:35:59.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7fbcb2ffd700 1 -- 172.21.15.79:0/1494348560 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4004230 con 0x7fbcb4069e90 2023-06-23T18:35:59.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7fbcb2ffd700 1 -- 172.21.15.79:0/1494348560 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca4004390 con 0x7fbcb4069e90 2023-06-23T18:35:59.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7fbcb2ffd700 1 -- 172.21.15.79:0/1494348560 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca40036a0 con 0x7fbcb4069e90 2023-06-23T18:35:59.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.734+0000 7f4c8d3c6700 1 -- start start 2023-06-23T18:35:59.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.735+0000 7f4c8d3c6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c8806e5f0 0x7f4c880b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.735+0000 7f4c8d3c6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c880c52f0 0x7f4c880b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.735+0000 7f4c8d3c6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c881608e0 0x7f4c882019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.735+0000 7f4c8d3c6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880b3840 con 0x7f4c880c52f0 2023-06-23T18:35:59.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.735+0000 7f4c8d3c6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880b1e10 con 0x7f4c881608e0 2023-06-23T18:35:59.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.735+0000 7f4c8d3c6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880b3a70 con 0x7f4c8806e5f0 2023-06-23T18:35:59.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7f4c86ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c8806e5f0 0x7f4c880b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.735+0000 7f1b48d89700 1 Processor -- start 2023-06-23T18:35:59.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.735+0000 7f1b48d89700 1 -- start start 2023-06-23T18:35:59.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7fd5ed7fa700 1 -- 172.21.15.79:0/2491090489 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd5e400b160 con 0x7fd5f00adce0 2023-06-23T18:35:59.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7f4c86ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c8806e5f0 0x7f4c880b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:60414/0 (socket says 172.21.15.79:60414) 2023-06-23T18:35:59.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7f4c86ffd700 1 -- 172.21.15.79:0/995975116 learned_addr learned my addr 172.21.15.79:0/995975116 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7fbcba2d1700 1 -- 172.21.15.79:0/1494348560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb4069e90 msgr2=0x7fbcb406a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7fbcba2d1700 1 --2- 172.21.15.79:0/1494348560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb4069e90 0x7fbcb406a310 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009840 tx=0x7fbca4009b50 comp rx=0 tx=0).stop 2023-06-23T18:35:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7fbcba2d1700 1 -- 172.21.15.79:0/1494348560 shutdown_connections 2023-06-23T18:35:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7fbcba2d1700 1 --2- 172.21.15.79:0/1494348560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb4069e90 0x7fbcb406a310 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7fbcba2d1700 1 --2- 172.21.15.79:0/1494348560 >> v2:172.21.15.79:3300/0 conn(0x7fbcb40c91c0 0x7fbcb4070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7fbcba2d1700 1 -- 172.21.15.79:0/1494348560 >> 172.21.15.79:0/1494348560 conn(0x7fbcb40cb1c0 msgr2=0x7fbcb40cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7f4c877fe700 1 --2- 172.21.15.79:0/995975116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c881608e0 0x7f4c882019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7fbcba2d1700 1 -- 172.21.15.79:0/1494348560 shutdown_connections 2023-06-23T18:35:59.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7f4c86ffd700 1 -- 172.21.15.79:0/995975116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c881608e0 msgr2=0x7f4c882019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7f4c86ffd700 1 --2- 172.21.15.79:0/995975116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c881608e0 0x7f4c882019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7f4c86ffd700 1 -- 172.21.15.79:0/995975116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c880c52f0 msgr2=0x7f4c880b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7f4c86ffd700 1 --2- 172.21.15.79:0/995975116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c880c52f0 0x7f4c880b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7fd5ed7fa700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd5dc06ad40 0x7fd5dc06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.736+0000 7f4c86ffd700 1 -- 172.21.15.79:0/995975116 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c780094a0 con 0x7f4c8806e5f0 2023-06-23T18:35:59.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f4c86ffd700 1 --2- 172.21.15.79:0/995975116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c8806e5f0 0x7f4c880b3300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000ccd0 tx=0x7f4c8000a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7fd5ed7fa700 1 -- 172.21.15.79:0/2491090489 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd5e4089dd0 con 0x7fd5f00adce0 2023-06-23T18:35:59.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7fd5effff700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd5dc06ad40 0x7fd5dc06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7fbcba2d1700 1 -- 172.21.15.79:0/1494348560 wait complete. 2023-06-23T18:35:59.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f4c6f7fe700 1 -- 172.21.15.79:0/995975116 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000ad30 con 0x7f4c8806e5f0 2023-06-23T18:35:59.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f3c267fc700 1 -- 172.21.15.79:0/3713069423 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3c2c01a400 con 0x7f3c30160590 2023-06-23T18:35:59.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f3c267fc700 1 --2- 172.21.15.79:0/3713069423 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3c1406ad30 0x7f3c1406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f3c267fc700 1 -- 172.21.15.79:0/3713069423 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3c2c0894c0 con 0x7f3c30160590 2023-06-23T18:35:59.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f4c6f7fe700 1 -- 172.21.15.79:0/995975116 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8000ae90 con 0x7f4c8806e5f0 2023-06-23T18:35:59.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f4c6f7fe700 1 -- 172.21.15.79:0/995975116 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80003c40 con 0x7f4c8806e5f0 2023-06-23T18:35:59.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/995975116 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c880b3d50 con 0x7f4c8806e5f0 2023-06-23T18:35:59.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/995975116 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c880b4310 con 0x7f4c8806e5f0 2023-06-23T18:35:59.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f1b48d89700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1b440c91c0 0x7f1b44070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f1b48d89700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1b440ccc90 0x7f1b44069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f1b48d89700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b44071180 con 0x7f1b4406a3d0 2023-06-23T18:35:59.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f1b48d89700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4416cae0 con 0x7f1b440c91c0 2023-06-23T18:35:59.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f1b48d89700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b440c3a40 con 0x7f1b440ccc90 2023-06-23T18:35:59.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f1b4259c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1b440c91c0 0x7f1b44070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f1b4259c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1b440c91c0 0x7f1b44070c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39420/0 (socket says 172.21.15.79:39420) 2023-06-23T18:35:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f1b4259c700 1 -- 172.21.15.79:0/1929051005 learned_addr learned my addr 172.21.15.79:0/1929051005 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f1b41d9b700 1 --2- 172.21.15.79:0/1929051005 >> v2:172.21.15.155:3300/0 conn(0x7f1b440ccc90 0x7f1b44069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f1b41d9b700 1 -- 172.21.15.79:0/1929051005 >> v2:172.21.15.79:3300/0 conn(0x7f1b440c91c0 msgr2=0x7f1b44070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f1b41d9b700 1 --2- 172.21.15.79:0/1929051005 >> v2:172.21.15.79:3300/0 conn(0x7f1b440c91c0 0x7f1b44070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f1b41d9b700 1 -- 172.21.15.79:0/1929051005 >> v1:172.21.15.155:6789/0 conn(0x7f1b4406a3d0 legacy=0x7f1b440c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.737+0000 7f1b41d9b700 1 -- 172.21.15.79:0/1929051005 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4406a810 con 0x7f1b440ccc90 2023-06-23T18:35:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.738+0000 7f1b41d9b700 1 --2- 172.21.15.79:0/1929051005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b440ccc90 0x7f1b44069e90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1b440cc1f0 tx=0x7f1b38009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e23ea1d2f2ed388d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.738+0000 7f3c357ef700 1 --2- 172.21.15.79:0/3713069423 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3c1406ad30 0x7f3c1406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.738+0000 7fd5effff700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd5dc06ad40 0x7fd5dc06d1f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00c5cc0 tx=0x7fd5e8009540 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.738+0000 7fbcba2d1700 1 Processor -- start 2023-06-23T18:35:59.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.739+0000 7f3c357ef700 1 --2- 172.21.15.79:0/3713069423 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3c1406ad30 0x7f3c1406d1e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c3015d4e0 tx=0x7f3c200090e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.739+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/995975116 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c880c37e0 con 0x7f4c8806e5f0 2023-06-23T18:35:59.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.739+0000 7efe7e7fc700 1 --2- 172.21.15.79:0/611144571 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe8006a5a0 0x7efe800c3dc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efe74009840 tx=0x7efe74009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5398bbdcd249c13b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.739+0000 7efe7dffb700 1 -- 172.21.15.79:0/611144571 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74004230 con 0x7efe8006a5a0 2023-06-23T18:35:59.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.739+0000 7efe7dffb700 1 -- 172.21.15.79:0/611144571 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe74004390 con 0x7efe8006a5a0 2023-06-23T18:35:59.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.739+0000 7efe7dffb700 1 -- 172.21.15.79:0/611144571 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe740036a0 con 0x7efe8006a5a0 2023-06-23T18:35:59.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.739+0000 7f8daa7fc700 1 -- 172.21.15.79:0/852375791 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8db0004c10 con 0x7f8db40cad80 2023-06-23T18:35:59.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.739+0000 7fbcba2d1700 1 -- start start 2023-06-23T18:35:59.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7f4c6f7fe700 1 -- 172.21.15.79:0/995975116 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c80004020 con 0x7f4c8806e5f0 2023-06-23T18:35:59.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7fbcba2d1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcb4069e90 0x7fbcb40c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7efe857cc700 1 -- 172.21.15.79:0/611144571 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe8006a5a0 msgr2=0x7efe800c3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7efe857cc700 1 --2- 172.21.15.79:0/611144571 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe8006a5a0 0x7efe800c3dc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efe74009840 tx=0x7efe74009b50 comp rx=0 tx=0).stop 2023-06-23T18:35:59.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7fbcba2d1700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb40add90 0x7fbcb4208c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7fbcba2d1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcb40c91c0 0x7fbcb40885b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7fbcba2d1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb415baf0 con 0x7fbcb4069e90 2023-06-23T18:35:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7fbcba2d1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb415aee0 con 0x7fbcb40c91c0 2023-06-23T18:35:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7fbcba2d1700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb415b1c0 con 0x7fbcb40add90 2023-06-23T18:35:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7f8daa7fc700 1 --2- 172.21.15.79:0/852375791 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8d9806ad30 0x7f8d9806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7f8daa7fc700 1 -- 172.21.15.79:0/852375791 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8db0088ed0 con 0x7f8db40cad80 2023-06-23T18:35:59.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7f8db9ae0700 1 --2- 172.21.15.79:0/852375791 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8d9806ad30 0x7f8d9806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7efe857cc700 1 -- 172.21.15.79:0/611144571 shutdown_connections 2023-06-23T18:35:59.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7efe857cc700 1 --2- 172.21.15.79:0/611144571 >> v2:172.21.15.155:3300/0 conn(0x7efe800c52f0 0x7efe800c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7efe857cc700 1 --2- 172.21.15.79:0/611144571 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe8006a5a0 0x7efe800c3dc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7efe857cc700 1 -- 172.21.15.79:0/611144571 >> 172.21.15.79:0/611144571 conn(0x7efe800cb1c0 msgr2=0x7efe800cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7fbcb886e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcb40c91c0 0x7fbcb40885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7f4c6f7fe700 1 --2- 172.21.15.79:0/995975116 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4c7406ad30 0x7f4c7406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7f4c6f7fe700 1 -- 172.21.15.79:0/995975116 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c80089590 con 0x7f4c8806e5f0 2023-06-23T18:35:59.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7f4c867fc700 1 --2- 172.21.15.79:0/995975116 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4c7406ad30 0x7f4c7406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7fbcb37fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb40add90 0x7fbcb4208c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7fbcb37fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb40add90 0x7fbcb4208c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:60438/0 (socket says 172.21.15.79:60438) 2023-06-23T18:35:59.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7fbcb37fe700 1 -- 172.21.15.79:0/2707218632 learned_addr learned my addr 172.21.15.79:0/2707218632 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7efe857cc700 1 -- 172.21.15.79:0/611144571 shutdown_connections 2023-06-23T18:35:59.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7efe857cc700 1 -- 172.21.15.79:0/611144571 wait complete. 2023-06-23T18:35:59.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.740+0000 7fbcb3fff700 1 --2- 172.21.15.79:0/2707218632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcb4069e90 0x7fbcb40c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.741+0000 7fbcb37fe700 1 -- 172.21.15.79:0/2707218632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcb40c91c0 msgr2=0x7fbcb40885b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.741+0000 7fbcb37fe700 1 --2- 172.21.15.79:0/2707218632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcb40c91c0 0x7fbcb40885b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.741+0000 7fbcb37fe700 1 -- 172.21.15.79:0/2707218632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcb4069e90 msgr2=0x7fbcb40c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.741+0000 7fbcb37fe700 1 --2- 172.21.15.79:0/2707218632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcb4069e90 0x7fbcb40c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.741+0000 7fbcb37fe700 1 -- 172.21.15.79:0/2707218632 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca40094a0 con 0x7fbcb40add90 2023-06-23T18:35:59.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.741+0000 7f8db9ae0700 1 --2- 172.21.15.79:0/852375791 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8d9806ad30 0x7f8d9806d1e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00abb0 tx=0x7f8dac009250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.741+0000 7f4c867fc700 1 --2- 172.21.15.79:0/995975116 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4c7406ad30 0x7f4c7406d1e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4c8806e3c0 tx=0x7f4c78010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.741+0000 7fbd067fc700 1 -- 172.21.15.79:0/3128999824 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbd101571b0 con 0x7fbd100aee30 2023-06-23T18:35:59.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.741+0000 7fbcb37fe700 1 --2- 172.21.15.79:0/2707218632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb40add90 0x7fbcb4208c50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbca800d800 tx=0x7fbca800db10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.742+0000 7efe857cc700 1 Processor -- start 2023-06-23T18:35:59.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.742+0000 7fd5ed7fa700 1 -- 172.21.15.79:0/2491090489 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd5e40550a0 con 0x7fd5f00adce0 2023-06-23T18:35:59.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.742+0000 7efe857cc700 1 -- start start 2023-06-23T18:35:59.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe857cc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe8006a5a0 0x7efe8014b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe857cc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe800c52f0 0x7efe800b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe857cc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe801608e0 0x7efe80160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe857cc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8014be90 con 0x7efe801608e0 2023-06-23T18:35:59.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe857cc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe800c5fa0 con 0x7efe8006a5a0 2023-06-23T18:35:59.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe857cc700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe800c6280 con 0x7efe800c52f0 2023-06-23T18:35:59.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe7f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe801608e0 0x7efe80160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe7e7fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe800c52f0 0x7efe800b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe7f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe801608e0 0x7efe80160e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39442/0 (socket says 172.21.15.79:39442) 2023-06-23T18:35:59.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe7effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe8006a5a0 0x7efe8014b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe7f7fe700 1 -- 172.21.15.79:0/1980295913 learned_addr learned my addr 172.21.15.79:0/1980295913 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe7f7fe700 1 -- 172.21.15.79:0/1980295913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe800c52f0 msgr2=0x7efe800b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe7f7fe700 1 --2- 172.21.15.79:0/1980295913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe800c52f0 0x7efe800b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe7f7fe700 1 -- 172.21.15.79:0/1980295913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe8006a5a0 msgr2=0x7efe8014b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe7f7fe700 1 --2- 172.21.15.79:0/1980295913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe8006a5a0 0x7efe8014b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe7f7fe700 1 -- 172.21.15.79:0/1980295913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe740094a0 con 0x7efe801608e0 2023-06-23T18:35:59.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.743+0000 7efe7f7fe700 1 --2- 172.21.15.79:0/1980295913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe801608e0 0x7efe80160e30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe7800efc0 tx=0x7efe7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.744+0000 7efe677fe700 1 -- 172.21.15.79:0/1980295913 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78003c20 con 0x7efe801608e0 2023-06-23T18:35:59.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.744+0000 7f8daa7fc700 1 -- 172.21.15.79:0/852375791 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8db00540f0 con 0x7f8db40cad80 2023-06-23T18:35:59.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.744+0000 7efe677fe700 1 -- 172.21.15.79:0/1980295913 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7800f040 con 0x7efe801608e0 2023-06-23T18:35:59.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.744+0000 7efe677fe700 1 -- 172.21.15.79:0/1980295913 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78009e30 con 0x7efe801608e0 2023-06-23T18:35:59.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.744+0000 7efe857cc700 1 -- 172.21.15.79:0/1980295913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe800c6560 con 0x7efe801608e0 2023-06-23T18:35:59.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.744+0000 7efe857cc700 1 -- 172.21.15.79:0/1980295913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe800c6b20 con 0x7efe801608e0 2023-06-23T18:35:59.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.745+0000 7fbcb0ff9700 1 -- 172.21.15.79:0/2707218632 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80125f0 con 0x7fbcb40add90 2023-06-23T18:35:59.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.745+0000 7fbcb0ff9700 1 -- 172.21.15.79:0/2707218632 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca8012750 con 0x7fbcb40add90 2023-06-23T18:35:59.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.745+0000 7fbcba2d1700 1 -- 172.21.15.79:0/2707218632 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb415b4a0 con 0x7fbcb40add90 2023-06-23T18:35:59.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.745+0000 7fbcb0ff9700 1 -- 172.21.15.79:0/2707218632 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8012ae0 con 0x7fbcb40add90 2023-06-23T18:35:59.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.745+0000 7fbcba2d1700 1 -- 172.21.15.79:0/2707218632 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb40c6120 con 0x7fbcb40add90 2023-06-23T18:35:59.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.746+0000 7efe857cc700 1 -- 172.21.15.79:0/1980295913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe800c6d10 con 0x7efe801608e0 2023-06-23T18:35:59.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.746+0000 7efe677fe700 1 -- 172.21.15.79:0/1980295913 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efe7800c880 con 0x7efe801608e0 2023-06-23T18:35:59.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.746+0000 7efe677fe700 1 --2- 172.21.15.79:0/1980295913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7efe6c06ad30 0x7efe6c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.745+0000 7f4c6f7fe700 1 -- 172.21.15.79:0/995975116 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4c800547b0 con 0x7f4c8806e5f0 2023-06-23T18:35:59.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.746+0000 7efe677fe700 1 -- 172.21.15.79:0/1980295913 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efe78089690 con 0x7efe801608e0 2023-06-23T18:35:59.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.747+0000 7efe7effd700 1 --2- 172.21.15.79:0/1980295913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7efe6c06ad30 0x7efe6c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.747+0000 7f3c267fc700 1 -- 172.21.15.79:0/3713069423 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3c2c0544d0 con 0x7f3c30160590 2023-06-23T18:35:59.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.747+0000 7efe7effd700 1 --2- 172.21.15.79:0/1980295913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7efe6c06ad30 0x7efe6c06d1e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efe70009960 tx=0x7efe700090e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.747+0000 7fbcb0ff9700 1 -- 172.21.15.79:0/2707218632 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbca8012c40 con 0x7fbcb40add90 2023-06-23T18:35:59.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.747+0000 7fbcb0ff9700 1 --2- 172.21.15.79:0/2707218632 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbc9c06ae40 0x7fbc9c06d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.748+0000 7fbcb0ff9700 1 -- 172.21.15.79:0/2707218632 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbca8089430 con 0x7fbcb40add90 2023-06-23T18:35:59.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.751+0000 7efe677fe700 1 -- 172.21.15.79:0/1980295913 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efe78054790 con 0x7efe801608e0 2023-06-23T18:35:59.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.752+0000 7f1b4159a700 1 -- 172.21.15.79:0/1929051005 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38022640 con 0x7f1b440ccc90 2023-06-23T18:35:59.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.752+0000 7f1b4159a700 1 -- 172.21.15.79:0/1929051005 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b380227a0 con 0x7f1b440ccc90 2023-06-23T18:35:59.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.752+0000 7f1b4159a700 1 -- 172.21.15.79:0/1929051005 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38022b30 con 0x7f1b440ccc90 2023-06-23T18:35:59.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.753+0000 7f1b48d89700 1 -- 172.21.15.79:0/1929051005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b440ccc90 msgr2=0x7f1b44069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.753+0000 7f1b48d89700 1 --2- 172.21.15.79:0/1929051005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b440ccc90 0x7f1b44069e90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1b440cc1f0 tx=0x7f1b38009b20 comp rx=0 tx=0).stop 2023-06-23T18:35:59.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.753+0000 7f1b48d89700 1 -- 172.21.15.79:0/1929051005 shutdown_connections 2023-06-23T18:35:59.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.753+0000 7f1b48d89700 1 --2- 172.21.15.79:0/1929051005 >> v2:172.21.15.79:3300/0 conn(0x7f1b440c91c0 0x7f1b44070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.753+0000 7f1b48d89700 1 --2- 172.21.15.79:0/1929051005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b440ccc90 0x7f1b44069e90 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1b440cc1f0 tx=0x7f1b38009b20 comp rx=0 tx=0).stop 2023-06-23T18:35:59.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.753+0000 7f1b48d89700 1 -- 172.21.15.79:0/1929051005 >> 172.21.15.79:0/1929051005 conn(0x7f1b440cb1c0 msgr2=0x7f1b440cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:59.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.752+0000 7fbcb3fff700 1 --2- 172.21.15.79:0/2707218632 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbc9c06ae40 0x7fbc9c06d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.758+0000 7fbcb3fff700 1 --2- 172.21.15.79:0/2707218632 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbc9c06ae40 0x7fbc9c06d2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbcac002030 tx=0x7fbcac001fc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.758+0000 7f1b48d89700 1 -- 172.21.15.79:0/1929051005 shutdown_connections 2023-06-23T18:35:59.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.758+0000 7f1b48d89700 1 -- 172.21.15.79:0/1929051005 wait complete. 2023-06-23T18:35:59.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.760+0000 7f1b48d89700 1 Processor -- start 2023-06-23T18:35:59.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.761+0000 7f1b48d89700 1 -- start start 2023-06-23T18:35:59.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.762+0000 7f1b48d89700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b4406a3d0 0x7f1b44169250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.762+0000 7f1b48d89700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b440c91c0 0x7f1b441523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.762+0000 7f1b48d89700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b44169790 0x7f1b44152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.762+0000 7f1b48d89700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b440ae930 con 0x7f1b4406a3d0 2023-06-23T18:35:59.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.762+0000 7f1b48d89700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b44169c30 con 0x7f1b440c91c0 2023-06-23T18:35:59.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.762+0000 7f1b48d89700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b44169f10 con 0x7f1b44169790 2023-06-23T18:35:59.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.762+0000 7f1b41d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b440c91c0 0x7f1b441523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.762+0000 7f1b41d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b440c91c0 0x7f1b441523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:45676/0 (socket says 172.21.15.79:45676) 2023-06-23T18:35:59.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.762+0000 7f1b41d9b700 1 -- 172.21.15.79:0/1877846461 learned_addr learned my addr 172.21.15.79:0/1877846461 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:35:59.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.763+0000 7f1b41d9b700 1 -- 172.21.15.79:0/1877846461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b44169790 msgr2=0x7f1b44152920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-23T18:35:59.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.763+0000 7f1b41d9b700 1 --2- 172.21.15.79:0/1877846461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b44169790 0x7f1b44152920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.763+0000 7f1b41d9b700 1 -- 172.21.15.79:0/1877846461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b4406a3d0 msgr2=0x7f1b44169250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:59.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.763+0000 7f1b41d9b700 1 --2- 172.21.15.79:0/1877846461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b4406a3d0 0x7f1b44169250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:59.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.763+0000 7f1b41d9b700 1 -- 172.21.15.79:0/1877846461 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b380094a0 con 0x7f1b440c91c0 2023-06-23T18:35:59.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.763+0000 7f1b41d9b700 1 --2- 172.21.15.79:0/1877846461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b440c91c0 0x7f1b441523e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b38024000 tx=0x7f1b38004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.766+0000 7f1b32ffd700 1 -- 172.21.15.79:0/1877846461 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38003b30 con 0x7f1b440c91c0 2023-06-23T18:35:59.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.766+0000 7f1b32ffd700 1 -- 172.21.15.79:0/1877846461 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b38030070 con 0x7f1b440c91c0 2023-06-23T18:35:59.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.766+0000 7f1b32ffd700 1 -- 172.21.15.79:0/1877846461 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38037480 con 0x7f1b440c91c0 2023-06-23T18:35:59.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.766+0000 7f1b48d89700 1 -- 172.21.15.79:0/1877846461 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4416a1f0 con 0x7f1b440c91c0 2023-06-23T18:35:59.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.766+0000 7f1b48d89700 1 -- 172.21.15.79:0/1877846461 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b44167220 con 0x7f1b440c91c0 2023-06-23T18:35:59.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.768+0000 7f1b32ffd700 1 -- 172.21.15.79:0/1877846461 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1b3802b430 con 0x7f1b440c91c0 2023-06-23T18:35:59.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.769+0000 7f1b32ffd700 1 --2- 172.21.15.79:0/1877846461 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1b2406ad30 0x7f1b2406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:59.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.769+0000 7f1b32ffd700 1 -- 172.21.15.79:0/1877846461 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1b38032070 con 0x7f1b440c91c0 2023-06-23T18:35:59.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.770+0000 7f1b4259c700 1 --2- 172.21.15.79:0/1877846461 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1b2406ad30 0x7f1b2406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:59.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.770+0000 7f1b4259c700 1 --2- 172.21.15.79:0/1877846461 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1b2406ad30 0x7f1b2406d1e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400c260 tx=0x7f1b34008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:59.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.775+0000 7fbcba2d1700 1 -- 172.21.15.79:0/2707218632 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc940052f0 con 0x7fbcb40add90 2023-06-23T18:35:59.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.776+0000 7f1b48d89700 1 -- 172.21.15.79:0/1877846461 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b280052f0 con 0x7f1b440c91c0 2023-06-23T18:35:59.843 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.780+0000 7fbcb0ff9700 1 -- 172.21.15.79:0/2707218632 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbca80545c0 con 0x7fbcb40add90 2023-06-23T18:35:59.843 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:35:59.781+0000 7f1b32ffd700 1 -- 172.21.15.79:0/1877846461 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1b38067250 con 0x7f1b440c91c0 2023-06-23T18:36:00.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.127+0000 7f3c38254700 1 -- 172.21.15.79:0/3713069423 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3c3015b3d0 con 0x7f3c30160590 2023-06-23T18:36:00.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.127+0000 7f3c267fc700 1 -- 172.21.15.79:0/3713069423 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3c2c057af0 con 0x7f3c30160590 2023-06-23T18:36:00.135 INFO:teuthology.orchestra.run.smithi079.stdout:55834574853 2023-06-23T18:36:00.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.131+0000 7f3c38254700 1 -- 172.21.15.79:0/3713069423 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3c1406ad30 msgr2=0x7f3c1406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.131+0000 7f3c38254700 1 --2- 172.21.15.79:0/3713069423 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3c1406ad30 0x7f3c1406d1e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c3015d4e0 tx=0x7f3c200090e0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.131+0000 7f3c38254700 1 -- 172.21.15.79:0/3713069423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c30160590 msgr2=0x7f3c300cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.131+0000 7f3c38254700 1 --2- 172.21.15.79:0/3713069423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c30160590 0x7f3c300cca30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00b580 tx=0x7f3c2c00b890 comp rx=0 tx=0).stop 2023-06-23T18:36:00.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.132+0000 7f3c38254700 1 -- 172.21.15.79:0/3713069423 shutdown_connections 2023-06-23T18:36:00.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.132+0000 7f3c38254700 1 --2- 172.21.15.79:0/3713069423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c300b8080 0x7f3c300c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.132+0000 7f3c38254700 1 --2- 172.21.15.79:0/3713069423 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3c1406ad30 0x7f3c1406d1e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.132+0000 7f3c38254700 1 --2- 172.21.15.79:0/3713069423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c30160590 0x7f3c300cca30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.132+0000 7f3c38254700 1 --2- 172.21.15.79:0/3713069423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c30164e90 0x7f3c30201c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.132+0000 7f3c38254700 1 -- 172.21.15.79:0/3713069423 >> 172.21.15.79:0/3713069423 conn(0x7f3c3006b350 msgr2=0x7f3c300d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:00.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.132+0000 7f3c38254700 1 -- 172.21.15.79:0/3713069423 shutdown_connections 2023-06-23T18:36:00.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.132+0000 7f3c38254700 1 -- 172.21.15.79:0/3713069423 wait complete. 2023-06-23T18:36:00.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.136+0000 7efe857cc700 1 -- 172.21.15.79:0/1980295913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efe800c37e0 con 0x7efe801608e0 2023-06-23T18:36:00.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.136+0000 7efe677fe700 1 -- 172.21.15.79:0/1980295913 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efe78057db0 con 0x7efe801608e0 2023-06-23T18:36:00.145 INFO:teuthology.orchestra.run.smithi079.stdout:55834574853 2023-06-23T18:36:00.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.140+0000 7fbd18079700 1 -- 172.21.15.79:0/3128999824 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbd10157590 con 0x7fbd100aee30 2023-06-23T18:36:00.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.140+0000 7efe857cc700 1 -- 172.21.15.79:0/1980295913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7efe6c06ad30 msgr2=0x7efe6c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.140+0000 7efe857cc700 1 --2- 172.21.15.79:0/1980295913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7efe6c06ad30 0x7efe6c06d1e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efe70009960 tx=0x7efe700090e0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.141+0000 7efe857cc700 1 -- 172.21.15.79:0/1980295913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe801608e0 msgr2=0x7efe80160e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.141+0000 7efe857cc700 1 --2- 172.21.15.79:0/1980295913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe801608e0 0x7efe80160e30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe7800efc0 tx=0x7efe7800c3a0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.141+0000 7efe857cc700 1 -- 172.21.15.79:0/1980295913 shutdown_connections 2023-06-23T18:36:00.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.141+0000 7efe857cc700 1 --2- 172.21.15.79:0/1980295913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe801608e0 0x7efe80160e30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.141+0000 7efe857cc700 1 --2- 172.21.15.79:0/1980295913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7efe6c06ad30 0x7efe6c06d1e0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efe70009960 tx=0x7efe700090e0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.141+0000 7efe857cc700 1 --2- 172.21.15.79:0/1980295913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe8006a5a0 0x7efe8014b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.141+0000 7efe857cc700 1 --2- 172.21.15.79:0/1980295913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe800c52f0 0x7efe800b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.141+0000 7efe857cc700 1 -- 172.21.15.79:0/1980295913 >> 172.21.15.79:0/1980295913 conn(0x7efe800cb1c0 msgr2=0x7efe8015e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:00.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.141+0000 7fbd067fc700 1 -- 172.21.15.79:0/3128999824 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbd10157590 con 0x7fbd100aee30 2023-06-23T18:36:00.150 INFO:teuthology.orchestra.run.smithi079.stdout:55834574852 2023-06-23T18:36:00.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.141+0000 7efe857cc700 1 -- 172.21.15.79:0/1980295913 shutdown_connections 2023-06-23T18:36:00.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.141+0000 7f8dbbd44700 1 -- 172.21.15.79:0/852375791 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f8db40ca5a0 con 0x7f8db40cad80 2023-06-23T18:36:00.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.141+0000 7efe857cc700 1 -- 172.21.15.79:0/1980295913 wait complete. 2023-06-23T18:36:00.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.142+0000 7f8daa7fc700 1 -- 172.21.15.79:0/852375791 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8db0018020 con 0x7f8db40cad80 2023-06-23T18:36:00.153 INFO:teuthology.orchestra.run.smithi079.stdout:55834574853 2023-06-23T18:36:00.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.144+0000 7fbd18079700 1 -- 172.21.15.79:0/3128999824 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbcf406ace0 msgr2=0x7fbcf406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.144+0000 7fbd18079700 1 --2- 172.21.15.79:0/3128999824 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbcf406ace0 0x7fbcf406d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd00006fd0 tx=0x7fbd00006bc0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.144+0000 7fbd18079700 1 -- 172.21.15.79:0/3128999824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aee30 msgr2=0x7fbd1015eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.144+0000 7fbd18079700 1 --2- 172.21.15.79:0/3128999824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aee30 0x7fbd1015eed0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c014040 tx=0x7fbd0c004750 comp rx=0 tx=0).stop 2023-06-23T18:36:00.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.143+0000 7fd5f71f2700 1 -- 172.21.15.79:0/2491090489 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd5f006dbf0 con 0x7fd5f00adce0 2023-06-23T18:36:00.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.145+0000 7fd5ed7fa700 1 -- 172.21.15.79:0/2491090489 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd5e40586c0 con 0x7fd5f00adce0 2023-06-23T18:36:00.155 INFO:teuthology.orchestra.run.smithi079.stdout:55834574852 2023-06-23T18:36:00.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/995975116 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4c880c7ea0 con 0x7f4c8806e5f0 2023-06-23T18:36:00.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.145+0000 7f8dbbd44700 1 -- 172.21.15.79:0/852375791 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8d9806ad30 msgr2=0x7f8d9806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.145+0000 7f8dbbd44700 1 --2- 172.21.15.79:0/852375791 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8d9806ad30 0x7f8d9806d1e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00abb0 tx=0x7f8dac009250 comp rx=0 tx=0).stop 2023-06-23T18:36:00.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.145+0000 7f8dbbd44700 1 -- 172.21.15.79:0/852375791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8db40cad80 msgr2=0x7f8db414d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.145+0000 7f8dbbd44700 1 --2- 172.21.15.79:0/852375791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8db40cad80 0x7f8db414d3e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8db000c700 tx=0x7f8db000cac0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7f8dbbd44700 1 -- 172.21.15.79:0/852375791 shutdown_connections 2023-06-23T18:36:00.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7f8dbbd44700 1 --2- 172.21.15.79:0/852375791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8db40cad80 0x7f8db414d3e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7f8dbbd44700 1 --2- 172.21.15.79:0/852375791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8db406b0b0 0x7f8db40b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7f8dbbd44700 1 --2- 172.21.15.79:0/852375791 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8d9806ad30 0x7f8d9806d1e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7f8dbbd44700 1 --2- 172.21.15.79:0/852375791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8db4070b10 0x7f8db40c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7f8dbbd44700 1 -- 172.21.15.79:0/852375791 >> 172.21.15.79:0/852375791 conn(0x7f8db40b17e0 msgr2=0x7f8db4162020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:00.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7fbd18079700 1 -- 172.21.15.79:0/3128999824 shutdown_connections 2023-06-23T18:36:00.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7fbd18079700 1 --2- 172.21.15.79:0/3128999824 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbcf406ace0 0x7fbcf406d190 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7fbd18079700 1 --2- 172.21.15.79:0/3128999824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd100b1270 0x7fbd1015f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7fbd18079700 1 --2- 172.21.15.79:0/3128999824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd10071490 0x7fbd100c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7fbd18079700 1 --2- 172.21.15.79:0/3128999824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aee30 0x7fbd1015eed0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7fbd18079700 1 -- 172.21.15.79:0/3128999824 >> 172.21.15.79:0/3128999824 conn(0x7fbd1006d660 msgr2=0x7fbd100cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:00.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7f4c6f7fe700 1 -- 172.21.15.79:0/995975116 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4c80057dd0 con 0x7f4c8806e5f0 2023-06-23T18:36:00.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.146+0000 7f8dbbd44700 1 -- 172.21.15.79:0/852375791 shutdown_connections 2023-06-23T18:36:00.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.147+0000 7f8dbbd44700 1 -- 172.21.15.79:0/852375791 wait complete. 2023-06-23T18:36:00.160 INFO:teuthology.orchestra.run.smithi079.stdout:55834574854 2023-06-23T18:36:00.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.148+0000 7fbd18079700 1 -- 172.21.15.79:0/3128999824 shutdown_connections 2023-06-23T18:36:00.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.148+0000 7fbd18079700 1 -- 172.21.15.79:0/3128999824 wait complete. 2023-06-23T18:36:00.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.151+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/995975116 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4c7406ad30 msgr2=0x7f4c7406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.151+0000 7f4c8d3c6700 1 --2- 172.21.15.79:0/995975116 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4c7406ad30 0x7f4c7406d1e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4c8806e3c0 tx=0x7f4c78010000 comp rx=0 tx=0).stop 2023-06-23T18:36:00.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.151+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/995975116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c8806e5f0 msgr2=0x7f4c880b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.151+0000 7f4c8d3c6700 1 --2- 172.21.15.79:0/995975116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c8806e5f0 0x7f4c880b3300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000ccd0 tx=0x7f4c8000a4d0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.151+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/995975116 shutdown_connections 2023-06-23T18:36:00.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.151+0000 7f4c8d3c6700 1 --2- 172.21.15.79:0/995975116 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4c7406ad30 0x7f4c7406d1e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.151+0000 7f4c8d3c6700 1 --2- 172.21.15.79:0/995975116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c881608e0 0x7f4c882019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.151+0000 7f4c8d3c6700 1 --2- 172.21.15.79:0/995975116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c8806e5f0 0x7f4c880b3300 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.151+0000 7f4c8d3c6700 1 --2- 172.21.15.79:0/995975116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c880c52f0 0x7f4c880b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.151+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/995975116 >> 172.21.15.79:0/995975116 conn(0x7f4c880cb1c0 msgr2=0x7f4c880c7790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:00.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.151+0000 7fd5f71f2700 1 -- 172.21.15.79:0/2491090489 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd5dc06ad40 msgr2=0x7fd5dc06d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.151+0000 7fd5f71f2700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd5dc06ad40 0x7fd5dc06d1f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00c5cc0 tx=0x7fd5e8009540 comp rx=0 tx=0).stop 2023-06-23T18:36:00.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.151+0000 7fd5f71f2700 1 -- 172.21.15.79:0/2491090489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5f00adce0 msgr2=0x7fd5f00c5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.151+0000 7fd5f71f2700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5f00adce0 0x7fd5f00c5780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400d370 tx=0x7fd5e4004750 comp rx=0 tx=0).stop 2023-06-23T18:36:00.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.152+0000 7fd5f71f2700 1 -- 172.21.15.79:0/2491090489 shutdown_connections 2023-06-23T18:36:00.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.152+0000 7fd5f71f2700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5f015ccc0 0x7fd5f006dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.152+0000 7fd5f71f2700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd5dc06ad40 0x7fd5dc06d1f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.152+0000 7fd5f71f2700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5f00adce0 0x7fd5f00c5780 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400d370 tx=0x7fd5e4004750 comp rx=0 tx=0).stop 2023-06-23T18:36:00.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.152+0000 7fd5f71f2700 1 --2- 172.21.15.79:0/2491090489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f01591a0 0x7fd5f00c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.152+0000 7fd5f71f2700 1 -- 172.21.15.79:0/2491090489 >> 172.21.15.79:0/2491090489 conn(0x7fd5f00cb1c0 msgr2=0x7fd5f00cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:00.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.152+0000 7fd5f71f2700 1 -- 172.21.15.79:0/2491090489 shutdown_connections 2023-06-23T18:36:00.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.152+0000 7fd5f71f2700 1 -- 172.21.15.79:0/2491090489 wait complete. 2023-06-23T18:36:00.165 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-06-23T18:36:00.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.152+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/995975116 shutdown_connections 2023-06-23T18:36:00.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.152+0000 7f4c8d3c6700 1 -- 172.21.15.79:0/995975116 wait complete. 2023-06-23T18:36:00.171 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2023-06-23T18:36:00.174 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-06-23T18:36:00.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.170+0000 7fbcba2d1700 1 -- 172.21.15.79:0/2707218632 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fbc94005870 con 0x7fbcb40add90 2023-06-23T18:36:00.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.171+0000 7fbcb0ff9700 1 -- 172.21.15.79:0/2707218632 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbca8057be0 con 0x7fbcb40add90 2023-06-23T18:36:00.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2023-06-23T18:36:00.176 INFO:teuthology.orchestra.run.smithi079.stdout:55834574854 2023-06-23T18:36:00.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.173+0000 7fbcba2d1700 1 -- 172.21.15.79:0/2707218632 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbc9c06ae40 msgr2=0x7fbc9c06d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.173+0000 7fbcba2d1700 1 --2- 172.21.15.79:0/2707218632 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbc9c06ae40 0x7fbc9c06d2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbcac002030 tx=0x7fbcac001fc0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.173+0000 7fbcba2d1700 1 -- 172.21.15.79:0/2707218632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb40add90 msgr2=0x7fbcb4208c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.173+0000 7fbcba2d1700 1 --2- 172.21.15.79:0/2707218632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb40add90 0x7fbcb4208c50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbca800d800 tx=0x7fbca800db10 comp rx=0 tx=0).stop 2023-06-23T18:36:00.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.173+0000 7fbcba2d1700 1 -- 172.21.15.79:0/2707218632 shutdown_connections 2023-06-23T18:36:00.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.173+0000 7fbcba2d1700 1 --2- 172.21.15.79:0/2707218632 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbc9c06ae40 0x7fbc9c06d2f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.173+0000 7fbcba2d1700 1 --2- 172.21.15.79:0/2707218632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcb40c91c0 0x7fbcb40885b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.173+0000 7fbcba2d1700 1 --2- 172.21.15.79:0/2707218632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcb4069e90 0x7fbcb40c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.173+0000 7fbcba2d1700 1 --2- 172.21.15.79:0/2707218632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb40add90 0x7fbcb4208c50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.173+0000 7fbcba2d1700 1 -- 172.21.15.79:0/2707218632 >> 172.21.15.79:0/2707218632 conn(0x7fbcb40cb1c0 msgr2=0x7fbcb4160730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.173+0000 7fbcba2d1700 1 -- 172.21.15.79:0/2707218632 shutdown_connections 2023-06-23T18:36:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.173+0000 7fbcba2d1700 1 -- 172.21.15.79:0/2707218632 wait complete. 2023-06-23T18:36:00.181 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2023-06-23T18:36:00.182 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-06-23T18:36:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.179+0000 7f1b48d89700 1 -- 172.21.15.79:0/1877846461 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1b28005870 con 0x7f1b440c91c0 2023-06-23T18:36:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.180+0000 7f1b32ffd700 1 -- 172.21.15.79:0/1877846461 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1b38026070 con 0x7f1b440c91c0 2023-06-23T18:36:00.185 INFO:teuthology.orchestra.run.smithi079.stdout:55834574854 2023-06-23T18:36:00.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.182+0000 7f1b48d89700 1 -- 172.21.15.79:0/1877846461 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1b2406ad30 msgr2=0x7f1b2406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.182+0000 7f1b48d89700 1 --2- 172.21.15.79:0/1877846461 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1b2406ad30 0x7f1b2406d1e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400c260 tx=0x7f1b34008040 comp rx=0 tx=0).stop 2023-06-23T18:36:00.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.182+0000 7f1b48d89700 1 -- 172.21.15.79:0/1877846461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b440c91c0 msgr2=0x7f1b441523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:00.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.182+0000 7f1b48d89700 1 --2- 172.21.15.79:0/1877846461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b440c91c0 0x7f1b441523e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b38024000 tx=0x7f1b38004750 comp rx=0 tx=0).stop 2023-06-23T18:36:00.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.182+0000 7f1b48d89700 1 -- 172.21.15.79:0/1877846461 shutdown_connections 2023-06-23T18:36:00.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.182+0000 7f1b48d89700 1 --2- 172.21.15.79:0/1877846461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b44169790 0x7f1b44152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.182+0000 7f1b48d89700 1 --2- 172.21.15.79:0/1877846461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b4406a3d0 0x7f1b44169250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.182+0000 7f1b48d89700 1 --2- 172.21.15.79:0/1877846461 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1b2406ad30 0x7f1b2406d1e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.182+0000 7f1b48d89700 1 --2- 172.21.15.79:0/1877846461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b440c91c0 0x7f1b441523e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:00.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.182+0000 7f1b48d89700 1 -- 172.21.15.79:0/1877846461 >> 172.21.15.79:0/1877846461 conn(0x7f1b440cb1c0 msgr2=0x7f1b4406d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:00.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.182+0000 7f1b48d89700 1 -- 172.21.15.79:0/1877846461 shutdown_connections 2023-06-23T18:36:00.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:00.182+0000 7f1b48d89700 1 -- 172.21.15.79:0/1877846461 wait complete. 2023-06-23T18:36:00.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-06-23T18:36:00.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-06-23T18:36:01.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-06-23T18:36:01.171 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-06-23T18:36:01.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-06-23T18:36:01.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-06-23T18:36:01.182 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-06-23T18:36:01.183 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-06-23T18:36:01.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-06-23T18:36:01.201 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-06-23T18:36:01.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.353+0000 7f4add1cb700 1 Processor -- start 2023-06-23T18:36:01.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.353+0000 7f4add1cb700 1 -- start start 2023-06-23T18:36:01.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.355+0000 7f4add1cb700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4ad806a5a0 0x7f4ad80adc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.355+0000 7f4add1cb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4ad815ccc0 0x7f4ad8158a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.355+0000 7f4add1cb700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad8159a60 con 0x7f4ad80cbcd0 2023-06-23T18:36:01.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.355+0000 7f4add1cb700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad80ae190 con 0x7f4ad815ccc0 2023-06-23T18:36:01.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.355+0000 7f4add1cb700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad8159780 con 0x7f4ad806a5a0 2023-06-23T18:36:01.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.356+0000 7f4ad659c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4ad806a5a0 0x7f4ad80adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.356+0000 7f4ad659c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4ad806a5a0 0x7f4ad80adc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:45682/0 (socket says 172.21.15.79:45682) 2023-06-23T18:36:01.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.356+0000 7f4ad659c700 1 -- 172.21.15.79:0/2375275555 learned_addr learned my addr 172.21.15.79:0/2375275555 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:01.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.356+0000 7f4ad759e700 1 --2- 172.21.15.79:0/2375275555 >> v2:172.21.15.79:3300/0 conn(0x7f4ad815ccc0 0x7f4ad8158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.357+0000 7f4ad759e700 1 -- 172.21.15.79:0/2375275555 >> v2:172.21.15.155:3300/0 conn(0x7f4ad806a5a0 msgr2=0x7f4ad80adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:01.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.357+0000 7f4ad759e700 1 --2- 172.21.15.79:0/2375275555 >> v2:172.21.15.155:3300/0 conn(0x7f4ad806a5a0 0x7f4ad80adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.357+0000 7f4ad759e700 1 -- 172.21.15.79:0/2375275555 >> v1:172.21.15.155:6789/0 conn(0x7f4ad80cbcd0 legacy=0x7f4ad8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:01.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.357+0000 7f4ad759e700 1 -- 172.21.15.79:0/2375275555 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad815c1e0 con 0x7f4ad815ccc0 2023-06-23T18:36:01.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.357+0000 7f4ad759e700 1 --2- 172.21.15.79:0/2375275555 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ad815ccc0 0x7f4ad8158a90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8070070 tx=0x7f4ac8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988375c65b4706dd server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.358+0000 7f4ad5d9b700 1 -- 172.21.15.79:0/2375275555 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8004230 con 0x7f4ad815ccc0 2023-06-23T18:36:01.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.358+0000 7f4ad5d9b700 1 -- 172.21.15.79:0/2375275555 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac8004390 con 0x7f4ad815ccc0 2023-06-23T18:36:01.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.358+0000 7f4ad5d9b700 1 -- 172.21.15.79:0/2375275555 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80036c0 con 0x7f4ad815ccc0 2023-06-23T18:36:01.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.359+0000 7f4add1cb700 1 -- 172.21.15.79:0/2375275555 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ad815ccc0 msgr2=0x7f4ad8158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.359+0000 7f4add1cb700 1 --2- 172.21.15.79:0/2375275555 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ad815ccc0 0x7f4ad8158a90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8070070 tx=0x7f4ac8009b20 comp rx=0 tx=0).stop 2023-06-23T18:36:01.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.360+0000 7f4add1cb700 1 -- 172.21.15.79:0/2375275555 shutdown_connections 2023-06-23T18:36:01.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.360+0000 7f4add1cb700 1 --2- 172.21.15.79:0/2375275555 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ad815ccc0 0x7f4ad8158a90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.360+0000 7f4add1cb700 1 --2- 172.21.15.79:0/2375275555 >> v2:172.21.15.155:3300/0 conn(0x7f4ad806a5a0 0x7f4ad80adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.360+0000 7f4add1cb700 1 -- 172.21.15.79:0/2375275555 >> 172.21.15.79:0/2375275555 conn(0x7f4ad80cb1c0 msgr2=0x7f4ad80cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.360+0000 7f4add1cb700 1 -- 172.21.15.79:0/2375275555 shutdown_connections 2023-06-23T18:36:01.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.360+0000 7f4add1cb700 1 -- 172.21.15.79:0/2375275555 wait complete. 2023-06-23T18:36:01.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.363+0000 7f4add1cb700 1 Processor -- start 2023-06-23T18:36:01.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.363+0000 7f4add1cb700 1 -- start start 2023-06-23T18:36:01.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.364+0000 7f4add1cb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad806a5a0 0x7f4ad806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.364+0000 7f4add1cb700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ad80cbcd0 0x7f4ad8146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.364+0000 7f4add1cb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ad815ccc0 0x7f4ad8147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.364+0000 7f4add1cb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad8147a50 con 0x7f4ad815ccc0 2023-06-23T18:36:01.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.364+0000 7f4add1cb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80c6430 con 0x7f4ad806a5a0 2023-06-23T18:36:01.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.364+0000 7f4add1cb700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80c58b0 con 0x7f4ad80cbcd0 2023-06-23T18:36:01.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.365+0000 7f4ad759e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ad815ccc0 0x7f4ad8147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.365+0000 7f4ad759e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ad815ccc0 0x7f4ad8147510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39466/0 (socket says 172.21.15.79:39466) 2023-06-23T18:36:01.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.365+0000 7f4ad759e700 1 -- 172.21.15.79:0/3754926951 learned_addr learned my addr 172.21.15.79:0/3754926951 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:01.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.366+0000 7f4ad759e700 1 -- 172.21.15.79:0/3754926951 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ad80cbcd0 msgr2=0x7f4ad8146fd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-23T18:36:01.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.366+0000 7f4ad759e700 1 --2- 172.21.15.79:0/3754926951 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ad80cbcd0 0x7f4ad8146fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.366+0000 7f4ad759e700 1 -- 172.21.15.79:0/3754926951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad806a5a0 msgr2=0x7f4ad806d660 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-23T18:36:01.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.366+0000 7f4ad759e700 1 --2- 172.21.15.79:0/3754926951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad806a5a0 0x7f4ad806d660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.366+0000 7f4ad759e700 1 -- 172.21.15.79:0/3754926951 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac80094a0 con 0x7f4ad815ccc0 2023-06-23T18:36:01.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.366+0000 7f4ad759e700 1 --2- 172.21.15.79:0/3754926951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ad815ccc0 0x7f4ad8147510 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ad806fe10 tx=0x7f4ac8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.367+0000 7f4ac77fe700 1 -- 172.21.15.79:0/3754926951 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8011630 con 0x7f4ad815ccc0 2023-06-23T18:36:01.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.367+0000 7f4ac77fe700 1 -- 172.21.15.79:0/3754926951 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac8011790 con 0x7f4ad815ccc0 2023-06-23T18:36:01.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.367+0000 7f4ac77fe700 1 -- 172.21.15.79:0/3754926951 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8011b20 con 0x7f4ad815ccc0 2023-06-23T18:36:01.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.367+0000 7f4add1cb700 1 -- 172.21.15.79:0/3754926951 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad80c5b90 con 0x7f4ad815ccc0 2023-06-23T18:36:01.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.367+0000 7f4add1cb700 1 -- 172.21.15.79:0/3754926951 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad80c6150 con 0x7f4ad815ccc0 2023-06-23T18:36:01.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.371+0000 7f4ac77fe700 1 -- 172.21.15.79:0/3754926951 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ac8018040 con 0x7f4ad815ccc0 2023-06-23T18:36:01.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.371+0000 7f4ac77fe700 1 --2- 172.21.15.79:0/3754926951 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4ac006af10 0x7f4ac006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.371+0000 7f4ac77fe700 1 -- 172.21.15.79:0/3754926951 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4ac808a5b0 con 0x7f4ad815ccc0 2023-06-23T18:36:01.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.376+0000 7f4add1cb700 1 -- 172.21.15.79:0/3754926951 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab40052f0 con 0x7f4ad815ccc0 2023-06-23T18:36:01.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.377+0000 7f4ad6d9d700 1 --2- 172.21.15.79:0/3754926951 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4ac006af10 0x7f4ac006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.378+0000 7f4ad6d9d700 1 --2- 172.21.15.79:0/3754926951 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4ac006af10 0x7f4ac006d3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4acc003d80 tx=0x7f4acc007a20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.385+0000 7f4ac77fe700 1 -- 172.21.15.79:0/3754926951 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4ac80556d0 con 0x7f4ad815ccc0 2023-06-23T18:36:01.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.418+0000 7fd04870a700 1 Processor -- start 2023-06-23T18:36:01.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.418+0000 7fd04870a700 1 -- start start 2023-06-23T18:36:01.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.419+0000 7fd04870a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd04006b0b0 0x7fd04006df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.419+0000 7fd04870a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd040070b10 0x7fd040070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.419+0000 7fd04870a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0400c1dd0 con 0x7fd0400cad80 2023-06-23T18:36:01.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.419+0000 7fd04870a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0400714d0 con 0x7fd04006b0b0 2023-06-23T18:36:01.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.419+0000 7fd04870a700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0400c2630 con 0x7fd040070b10 2023-06-23T18:36:01.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.420+0000 7fd045ca5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd04006b0b0 0x7fd04006df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.420+0000 7fd045ca5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd04006b0b0 0x7fd04006df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39472/0 (socket says 172.21.15.79:39472) 2023-06-23T18:36:01.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.420+0000 7fd045ca5700 1 -- 172.21.15.79:0/2800519572 learned_addr learned my addr 172.21.15.79:0/2800519572 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:01.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.420+0000 7fd046ca7700 1 --2- 172.21.15.79:0/2800519572 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd040070b10 0x7fd040070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.420+0000 7fd045ca5700 1 -- 172.21.15.79:0/2800519572 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd040070b10 msgr2=0x7fd040070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:01.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.420+0000 7fd045ca5700 1 --2- 172.21.15.79:0/2800519572 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd040070b10 0x7fd040070f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.420+0000 7fd045ca5700 1 -- 172.21.15.79:0/2800519572 >> v1:172.21.15.79:6789/0 conn(0x7fd0400cad80 legacy=0x7fd0400cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.420+0000 7fd045ca5700 1 -- 172.21.15.79:0/2800519572 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0400cb150 con 0x7fd04006b0b0 2023-06-23T18:36:01.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.420+0000 7fc6bdc98700 1 Processor -- start 2023-06-23T18:36:01.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.420+0000 7fc6bdc98700 1 -- start start 2023-06-23T18:36:01.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.420+0000 7fd045ca5700 1 --2- 172.21.15.79:0/2800519572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd04006b0b0 0x7fd04006df40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd030009920 tx=0x7fd030009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c8df91f8790d98 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.421+0000 7fd0454a4700 1 -- 172.21.15.79:0/2800519572 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030011830 con 0x7fd04006b0b0 2023-06-23T18:36:01.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.421+0000 7fd0454a4700 1 -- 172.21.15.79:0/2800519572 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd030011990 con 0x7fd04006b0b0 2023-06-23T18:36:01.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.421+0000 7fd0454a4700 1 -- 172.21.15.79:0/2800519572 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030011d20 con 0x7fd04006b0b0 2023-06-23T18:36:01.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.421+0000 7f95b8218700 1 Processor -- start 2023-06-23T18:36:01.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.421+0000 7f95b8218700 1 -- start start 2023-06-23T18:36:01.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.421+0000 7fc6bdc98700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc6b80706b0 0x7fc6b80ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.421+0000 7fc6bdc98700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b815d930 con 0x7fc6b8070230 2023-06-23T18:36:01.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.421+0000 7fc6bdc98700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b815cd20 con 0x7fc6b8159130 2023-06-23T18:36:01.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.421+0000 7fc6bdc98700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b815d000 con 0x7fc6b80706b0 2023-06-23T18:36:01.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fc6b77fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fc6b8159130 0x7fc6b80c4ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:50788/0 (socket says 172.21.15.79:50788) 2023-06-23T18:36:01.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fc6b77fe700 1 -- 172.21.15.79:0/2388888770 learned_addr learned my addr 172.21.15.79:0/2388888770 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:01.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fc6b7fff700 1 --2- 172.21.15.79:0/2388888770 >> v2:172.21.15.155:3300/0 conn(0x7fc6b80706b0 0x7fc6b80ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fd04870a700 1 -- 172.21.15.79:0/2800519572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd04006b0b0 msgr2=0x7fd04006df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fd04870a700 1 --2- 172.21.15.79:0/2800519572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd04006b0b0 0x7fd04006df40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd030009920 tx=0x7fd030009c30 comp rx=0 tx=0).stop 2023-06-23T18:36:01.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fd04870a700 1 -- 172.21.15.79:0/2800519572 shutdown_connections 2023-06-23T18:36:01.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fd04870a700 1 --2- 172.21.15.79:0/2800519572 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd040070b10 0x7fd040070f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fd04870a700 1 --2- 172.21.15.79:0/2800519572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd04006b0b0 0x7fd04006df40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fd04870a700 1 -- 172.21.15.79:0/2800519572 >> 172.21.15.79:0/2800519572 conn(0x7fd0400b17e0 msgr2=0x7fd04006cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fc6b67fc700 1 -- 172.21.15.79:0/2388888770 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099743187 0 0) 0x7fc6b815d930 con 0x7fc6b8070230 2023-06-23T18:36:01.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fc6b67fc700 1 -- 172.21.15.79:0/2388888770 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6a00057c0 con 0x7fc6b8070230 2023-06-23T18:36:01.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fc6b7fff700 1 -- 172.21.15.79:0/2388888770 >> v1:172.21.15.155:6789/0 conn(0x7fc6b8159130 legacy=0x7fc6b80c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fc6b7fff700 1 -- 172.21.15.79:0/2388888770 >> v1:172.21.15.79:6789/0 conn(0x7fc6b8070230 legacy=0x7fc6b80c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fc6b7fff700 1 -- 172.21.15.79:0/2388888770 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b815d2e0 con 0x7fc6b80706b0 2023-06-23T18:36:01.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.414+0000 7fd27df6b700 1 Processor -- start 2023-06-23T18:36:01.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.418+0000 7fd27df6b700 1 -- start start 2023-06-23T18:36:01.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.419+0000 7fd27df6b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2780894e0 0x7fd278069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.419+0000 7fd27df6b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd27806a3d0 0x7fd2780c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.419+0000 7fd27df6b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd27816db90 con 0x7fd2780add50 2023-06-23T18:36:01.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.419+0000 7fd27df6b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd27816d3e0 con 0x7fd27806a3d0 2023-06-23T18:36:01.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.419+0000 7fd27df6b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2780c91c0 con 0x7fd2780894e0 2023-06-23T18:36:01.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.420+0000 7fd277fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd27806a3d0 0x7fd2780c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.420+0000 7fd277fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd27806a3d0 0x7fd2780c42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39476/0 (socket says 172.21.15.79:39476) 2023-06-23T18:36:01.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.420+0000 7fd277fff700 1 -- 172.21.15.79:0/2768861704 learned_addr learned my addr 172.21.15.79:0/2768861704 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:01.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.422+0000 7fd276ffd700 1 --2- 172.21.15.79:0/2768861704 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2780894e0 0x7fd278069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7f95b8218700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f95b0159170 0x7f95b00cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7f95b8218700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f95b00cd900 0x7f95b00ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7fc6b7fff700 1 --2- 172.21.15.79:0/2388888770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6b80706b0 0x7fc6b80ca320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009d90 tx=0x7fc6b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19a2e04bc1dc15eb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7f95b8218700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b015cd50 con 0x7f95b00ce620 2023-06-23T18:36:01.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7f95b8218700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b00cd1e0 con 0x7f95b0159170 2023-06-23T18:36:01.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7f95b8218700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b00cdd80 con 0x7f95b00cd900 2023-06-23T18:36:01.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7fc6b67fc700 1 -- 172.21.15.79:0/2388888770 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b000e070 con 0x7fc6b80706b0 2023-06-23T18:36:01.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7fc6b67fc700 1 -- 172.21.15.79:0/2388888770 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b000bb20 con 0x7fc6b80706b0 2023-06-23T18:36:01.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7fc6b67fc700 1 -- 172.21.15.79:0/2388888770 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0012430 con 0x7fc6b80706b0 2023-06-23T18:36:01.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7f95b5fb4700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f95b0159170 0x7f95b00cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7f95b67b5700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f95b00ce620 0x7f95b0071490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:50802/0 (socket says 172.21.15.79:50802) 2023-06-23T18:36:01.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7f95b5fb4700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f95b0159170 0x7f95b00cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39490/0 (socket says 172.21.15.79:39490) 2023-06-23T18:36:01.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7f95b67b5700 1 -- 172.21.15.79:0/2593677289 learned_addr learned my addr 172.21.15.79:0/2593677289 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:01.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7f95b5fb4700 1 -- 172.21.15.79:0/2593677289 >> v2:172.21.15.155:3300/0 conn(0x7f95b00cd900 msgr2=0x7f95b00ce0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:36:01.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7f95b5fb4700 1 --2- 172.21.15.79:0/2593677289 >> v2:172.21.15.155:3300/0 conn(0x7f95b00cd900 0x7f95b00ce0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7f95b5fb4700 1 -- 172.21.15.79:0/2593677289 >> v1:172.21.15.155:6789/0 conn(0x7f95b00ce620 legacy=0x7f95b0071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:01.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7f95b5fb4700 1 -- 172.21.15.79:0/2593677289 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b00cea80 con 0x7f95b0159170 2023-06-23T18:36:01.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7f95b5fb4700 1 --2- 172.21.15.79:0/2593677289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95b0159170 0x7f95b00cd3c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f95b0070f20 tx=0x7f95ac009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e39782a251adc71 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7fd04870a700 1 -- 172.21.15.79:0/2800519572 shutdown_connections 2023-06-23T18:36:01.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7fd04870a700 1 -- 172.21.15.79:0/2800519572 wait complete. 2023-06-23T18:36:01.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2388888770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6b80706b0 msgr2=0x7fc6b80ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7fc6bdc98700 1 --2- 172.21.15.79:0/2388888770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6b80706b0 0x7fc6b80ca320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009d90 tx=0x7fc6b000b330 comp rx=0 tx=0).stop 2023-06-23T18:36:01.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7f95b4fb2700 1 -- 172.21.15.79:0/2593677289 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac004230 con 0x7f95b0159170 2023-06-23T18:36:01.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2388888770 shutdown_connections 2023-06-23T18:36:01.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7fc6bdc98700 1 --2- 172.21.15.79:0/2388888770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6b80706b0 0x7fc6b80ca320 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2388888770 >> 172.21.15.79:0/2388888770 conn(0x7fc6b80cf3d0 msgr2=0x7fc6b80cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7f95b4fb2700 1 -- 172.21.15.79:0/2593677289 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac004390 con 0x7f95b0159170 2023-06-23T18:36:01.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7f95b4fb2700 1 -- 172.21.15.79:0/2593677289 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0036c0 con 0x7f95b0159170 2023-06-23T18:36:01.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2388888770 shutdown_connections 2023-06-23T18:36:01.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2388888770 wait complete. 2023-06-23T18:36:01.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.425+0000 7f95b8218700 1 -- 172.21.15.79:0/2593677289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95b0159170 msgr2=0x7f95b00cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.425+0000 7f95b8218700 1 --2- 172.21.15.79:0/2593677289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95b0159170 0x7f95b00cd3c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f95b0070f20 tx=0x7f95ac009b20 comp rx=0 tx=0).stop 2023-06-23T18:36:01.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.425+0000 7f95b8218700 1 -- 172.21.15.79:0/2593677289 shutdown_connections 2023-06-23T18:36:01.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.425+0000 7f95b8218700 1 --2- 172.21.15.79:0/2593677289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95b0159170 0x7f95b00cd3c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.425+0000 7f95b8218700 1 --2- 172.21.15.79:0/2593677289 >> v2:172.21.15.155:3300/0 conn(0x7f95b00cd900 0x7f95b00ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.425+0000 7f95b8218700 1 -- 172.21.15.79:0/2593677289 >> 172.21.15.79:0/2593677289 conn(0x7f95b006d660 msgr2=0x7f95b006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.425+0000 7fd04870a700 1 Processor -- start 2023-06-23T18:36:01.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.426+0000 7f927f223700 1 Processor -- start 2023-06-23T18:36:01.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.426+0000 7fc6bdc98700 1 Processor -- start 2023-06-23T18:36:01.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.426+0000 7f927f223700 1 -- start start 2023-06-23T18:36:01.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.426+0000 7fd04870a700 1 -- start start 2023-06-23T18:36:01.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.426+0000 7fc6bdc98700 1 -- start start 2023-06-23T18:36:01.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7fd277fff700 1 -- 172.21.15.79:0/2768861704 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2780894e0 msgr2=0x7fd278069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:01.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7fd277fff700 1 --2- 172.21.15.79:0/2768861704 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2780894e0 0x7fd278069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7fd277fff700 1 -- 172.21.15.79:0/2768861704 >> v1:172.21.15.155:6789/0 conn(0x7fd2780add50 legacy=0x7fd27816ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7fd277fff700 1 -- 172.21.15.79:0/2768861704 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2780c3e50 con 0x7fd27806a3d0 2023-06-23T18:36:01.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7fd277fff700 1 --2- 172.21.15.79:0/2768861704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd27806a3d0 0x7fd2780c42e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd2700083f0 tx=0x7fd27000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4237ece1c334ab server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7fd2767fc700 1 -- 172.21.15.79:0/2768861704 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27000fe80 con 0x7fd27806a3d0 2023-06-23T18:36:01.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7fd2767fc700 1 -- 172.21.15.79:0/2768861704 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27000a2e0 con 0x7fd27806a3d0 2023-06-23T18:36:01.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.423+0000 7fd2767fc700 1 -- 172.21.15.79:0/2768861704 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd270003c20 con 0x7fd27806a3d0 2023-06-23T18:36:01.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7fd27df6b700 1 -- 172.21.15.79:0/2768861704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd27806a3d0 msgr2=0x7fd2780c42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7fd27df6b700 1 --2- 172.21.15.79:0/2768861704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd27806a3d0 0x7fd2780c42e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd2700083f0 tx=0x7fd27000f730 comp rx=0 tx=0).stop 2023-06-23T18:36:01.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7fd27df6b700 1 -- 172.21.15.79:0/2768861704 shutdown_connections 2023-06-23T18:36:01.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7fd27df6b700 1 --2- 172.21.15.79:0/2768861704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd27806a3d0 0x7fd2780c42e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7fd27df6b700 1 --2- 172.21.15.79:0/2768861704 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2780894e0 0x7fd278069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.424+0000 7fd27df6b700 1 -- 172.21.15.79:0/2768861704 >> 172.21.15.79:0/2768861704 conn(0x7fd2780cb1c0 msgr2=0x7fd2780cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.425+0000 7fd27df6b700 1 -- 172.21.15.79:0/2768861704 shutdown_connections 2023-06-23T18:36:01.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.425+0000 7fd27df6b700 1 -- 172.21.15.79:0/2768861704 wait complete. 2023-06-23T18:36:01.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.426+0000 7fd27df6b700 1 Processor -- start 2023-06-23T18:36:01.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.426+0000 7fd27df6b700 1 -- start start 2023-06-23T18:36:01.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7f7ea3878700 1 Processor -- start 2023-06-23T18:36:01.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7fd04870a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd04006b0b0 0x7fd040158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7fd04870a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd040070b10 0x7fd040144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7fd04870a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0400cad80 0x7fd040144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7fd04870a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0401469f0 con 0x7fd04006b0b0 2023-06-23T18:36:01.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7fd04870a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd040146130 con 0x7fd040070b10 2023-06-23T18:36:01.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7fd04870a700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0401463a0 con 0x7fd0400cad80 2023-06-23T18:36:01.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6bdc98700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6b8070230 0x7fc6b8153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6bdc98700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6b80706b0 0x7fc6b80c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6bdc98700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6b8159130 0x7fc6b81fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6bdc98700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b815a6f0 con 0x7fc6b80706b0 2023-06-23T18:36:01.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6bdc98700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b80b3650 con 0x7fc6b8070230 2023-06-23T18:36:01.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6bdc98700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b80b4410 con 0x7fc6b8159130 2023-06-23T18:36:01.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6b6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6b80706b0 0x7fc6b80c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6b6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6b80706b0 0x7fc6b80c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39504/0 (socket says 172.21.15.79:39504) 2023-06-23T18:36:01.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6b6ffd700 1 -- 172.21.15.79:0/2647280373 learned_addr learned my addr 172.21.15.79:0/2647280373 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:01.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6b77fe700 1 --2- 172.21.15.79:0/2647280373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6b8070230 0x7fc6b8153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7f927f223700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9278160830 0x7f92780adae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7f927f223700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f92780c3600 0x7f9278069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7f927f223700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927806a7f0 con 0x7f927806a3d0 2023-06-23T18:36:01.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7f927f223700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92780ae020 con 0x7f92780c3600 2023-06-23T18:36:01.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7f927f223700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92780cd030 con 0x7f9278160830 2023-06-23T18:36:01.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7f9277fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f92780c3600 0x7f9278069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7f9277fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f92780c3600 0x7f9278069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39496/0 (socket says 172.21.15.79:39496) 2023-06-23T18:36:01.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7f9277fff700 1 -- 172.21.15.79:0/3278377200 learned_addr learned my addr 172.21.15.79:0/3278377200 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:01.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7f927cfbf700 1 --2- 172.21.15.79:0/3278377200 >> v2:172.21.15.155:3300/0 conn(0x7f9278160830 0x7f92780adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7f7ea3878700 1 -- start start 2023-06-23T18:36:01.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7fd27df6b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd27806a3d0 0x7fd27815e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7fd27df6b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2780894e0 0x7fd278165850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7fd27df6b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2780add50 0x7fd278165d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7fd27df6b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27815e770 con 0x7fd27806a3d0 2023-06-23T18:36:01.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7fd27df6b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2781662d0 con 0x7fd2780add50 2023-06-23T18:36:01.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.427+0000 7fd27df6b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd278167120 con 0x7fd2780894e0 2023-06-23T18:36:01.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fd2777fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd27806a3d0 0x7fd27815e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fd276ffd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2780894e0 0x7fd278165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fd2777fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd27806a3d0 0x7fd27815e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39498/0 (socket says 172.21.15.79:39498) 2023-06-23T18:36:01.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fd2777fe700 1 -- 172.21.15.79:0/876936772 learned_addr learned my addr 172.21.15.79:0/876936772 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:01.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fd046ca7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0400cad80 0x7fd040144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fd046ca7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0400cad80 0x7fd040144680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:60504/0 (socket says 172.21.15.79:60504) 2023-06-23T18:36:01.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fd046ca7700 1 -- 172.21.15.79:0/4157704538 learned_addr learned my addr 172.21.15.79:0/4157704538 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:01.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6b6ffd700 1 -- 172.21.15.79:0/2647280373 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6b8159130 msgr2=0x7fc6b81fa1e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-23T18:36:01.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6b6ffd700 1 --2- 172.21.15.79:0/2647280373 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6b8159130 0x7fc6b81fa1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6b6ffd700 1 -- 172.21.15.79:0/2647280373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6b8070230 msgr2=0x7fc6b8153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6b6ffd700 1 --2- 172.21.15.79:0/2647280373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6b8070230 0x7fc6b8153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fc6b6ffd700 1 -- 172.21.15.79:0/2647280373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b0009a70 con 0x7fc6b80706b0 2023-06-23T18:36:01.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7f92777fe700 1 -- 172.21.15.79:0/3278377200 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2070506260 0 0) 0x7f927806a7f0 con 0x7f927806a3d0 2023-06-23T18:36:01.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7f92777fe700 1 -- 172.21.15.79:0/3278377200 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92600035b0 con 0x7f927806a3d0 2023-06-23T18:36:01.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fd0464a6700 1 --2- 172.21.15.79:0/4157704538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd04006b0b0 0x7fd040158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fd2777fe700 1 -- 172.21.15.79:0/876936772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2780894e0 msgr2=0x7fd278165850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fd2777fe700 1 --2- 172.21.15.79:0/876936772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2780894e0 0x7fd278165850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fd2777fe700 1 -- 172.21.15.79:0/876936772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2780add50 msgr2=0x7fd278165d90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-23T18:36:01.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7fd2777fe700 1 --2- 172.21.15.79:0/876936772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2780add50 0x7fd278165d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7fd2777fe700 1 -- 172.21.15.79:0/876936772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27000ab50 con 0x7fd27806a3d0 2023-06-23T18:36:01.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.428+0000 7f92777fe700 1 -- 172.21.15.79:0/3278377200 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 540235256 0 0) 0x7f92600035b0 con 0x7f927806a3d0 2023-06-23T18:36:01.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f92777fe700 1 -- 172.21.15.79:0/3278377200 >> v2:172.21.15.155:3300/0 conn(0x7f9278160830 msgr2=0x7f92780adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:01.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f92777fe700 1 --2- 172.21.15.79:0/3278377200 >> v2:172.21.15.155:3300/0 conn(0x7f9278160830 0x7f92780adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f92777fe700 1 -- 172.21.15.79:0/3278377200 >> v2:172.21.15.79:3300/0 conn(0x7f92780c3600 msgr2=0x7f9278069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:01.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f92777fe700 1 --2- 172.21.15.79:0/3278377200 >> v2:172.21.15.79:3300/0 conn(0x7f92780c3600 0x7f9278069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f92777fe700 1 -- 172.21.15.79:0/3278377200 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9278161c10 con 0x7f927806a3d0 2023-06-23T18:36:01.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f92777fe700 1 -- 172.21.15.79:0/3278377200 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f926c001670 con 0x7f927806a3d0 2023-06-23T18:36:01.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f92777fe700 1 -- 172.21.15.79:0/3278377200 >> v1:172.21.15.155:6789/0 conn(0x7f927806a3d0 legacy=0x7f9278070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f92777fe700 1 --2- 172.21.15.79:0/3278377200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9260003a50 0x7f9260003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f92777fe700 1 --2- 172.21.15.79:0/3278377200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260004850 0x7f9260004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f92777fe700 1 --2- 172.21.15.79:0/3278377200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9260005200 0x7f926000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f92777fe700 1 -- 172.21.15.79:0/3278377200 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92780ae020 con 0x7f9260005200 2023-06-23T18:36:01.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f92777fe700 1 -- 172.21.15.79:0/3278377200 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92780cd030 con 0x7f9260004850 2023-06-23T18:36:01.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f92777fe700 1 -- 172.21.15.79:0/3278377200 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926000bfd0 con 0x7f9260003a50 2023-06-23T18:36:01.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7fc6b6ffd700 1 --2- 172.21.15.79:0/2647280373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6b80706b0 0x7fc6b80c8000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800c520 tx=0x7fc6a800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f9277fff700 1 --2- 172.21.15.79:0/3278377200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260004850 0x7f9260004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f927cfbf700 1 --2- 172.21.15.79:0/3278377200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9260003a50 0x7f9260003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7fd2777fe700 1 --2- 172.21.15.79:0/876936772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd27806a3d0 0x7fd27815e230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd26800c480 tx=0x7fd26800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7fd0464a6700 1 -- 172.21.15.79:0/4157704538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0400cad80 msgr2=0x7fd040144680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7fd0464a6700 1 --2- 172.21.15.79:0/4157704538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0400cad80 0x7fd040144680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7fd0464a6700 1 -- 172.21.15.79:0/4157704538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd040070b10 msgr2=0x7fd040144140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-23T18:36:01.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7fd0464a6700 1 --2- 172.21.15.79:0/4157704538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd040070b10 0x7fd040144140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7fd0464a6700 1 -- 172.21.15.79:0/4157704538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd030009550 con 0x7fd04006b0b0 2023-06-23T18:36:01.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f7ea3878700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7e9c0cbcd0 0x7f7e9c0adce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f7ea3878700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7e9c15ccc0 0x7f7e9c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f7ea3878700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e9c06a3d0 0x7f7e9c158a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f7ea3878700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c06a850 con 0x7f7e9c0cbcd0 2023-06-23T18:36:01.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f7ea3878700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c159a60 con 0x7f7e9c15ccc0 2023-06-23T18:36:01.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f7ea3878700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c159780 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7fd25ffff700 1 -- 172.21.15.79:0/876936772 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26800cea0 con 0x7fd27806a3d0 2023-06-23T18:36:01.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7fd25ffff700 1 -- 172.21.15.79:0/876936772 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2680052d0 con 0x7fd27806a3d0 2023-06-23T18:36:01.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7fd25ffff700 1 -- 172.21.15.79:0/876936772 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd268003680 con 0x7fd27806a3d0 2023-06-23T18:36:01.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f927d7c0700 1 --2- 172.21.15.79:0/3278377200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9260005200 0x7f926000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f9277fff700 1 -- 172.21.15.79:0/3278377200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9260003a50 msgr2=0x7f9260003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:01.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f9277fff700 1 --2- 172.21.15.79:0/3278377200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9260003a50 0x7f9260003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f9277fff700 1 -- 172.21.15.79:0/3278377200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9260005200 msgr2=0x7f926000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:01.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f9277fff700 1 --2- 172.21.15.79:0/3278377200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9260005200 0x7f926000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f9277fff700 1 -- 172.21.15.79:0/3278377200 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92600035b0 con 0x7f9260004850 2023-06-23T18:36:01.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7f927d7c0700 1 -- 172.21.15.79:0/3278377200 reap_dead start 2023-06-23T18:36:01.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7f7ea1e15700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e9c06a3d0 0x7f7e9c158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7f7ea1e15700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e9c06a3d0 0x7f7e9c158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:60528/0 (socket says 172.21.15.79:60528) 2023-06-23T18:36:01.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7f7ea1e15700 1 -- 172.21.15.79:0/1536520635 learned_addr learned my addr 172.21.15.79:0/1536520635 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:01.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7fd27df6b700 1 -- 172.21.15.79:0/876936772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd278166510 con 0x7fd27806a3d0 2023-06-23T18:36:01.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.429+0000 7fd27df6b700 1 -- 172.21.15.79:0/876936772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd278166ad0 con 0x7fd27806a3d0 2023-06-23T18:36:01.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7fd0464a6700 1 --2- 172.21.15.79:0/4157704538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd04006b0b0 0x7fd040158820 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00c440 tx=0x7fd03c00c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7f7ea0e13700 1 --2- 172.21.15.79:0/1536520635 >> v2:172.21.15.155:3300/0 conn(0x7f7e9c15ccc0 0x7f7e9c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7f7ea1614700 1 --2- 172.21.15.79:0/1536520635 >> v2:172.21.15.79:3300/0 conn(0x7f7e9c0cbcd0 0x7f7e9c0adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7fd036ffd700 1 -- 172.21.15.79:0/4157704538 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c00cea0 con 0x7fd04006b0b0 2023-06-23T18:36:01.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7fd036ffd700 1 -- 172.21.15.79:0/4157704538 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c0052d0 con 0x7fd04006b0b0 2023-06-23T18:36:01.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7fd036ffd700 1 -- 172.21.15.79:0/4157704538 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c006860 con 0x7fd04006b0b0 2023-06-23T18:36:01.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7f9277fff700 1 --2- 172.21.15.79:0/3278377200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260004850 0x7f9260004cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9268009be0 tx=0x7f9260005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5d1652d5550a168 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7f92777fe700 1 -- 172.21.15.79:0/3278377200 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926800e780 con 0x7f9260004850 2023-06-23T18:36:01.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.431+0000 7f92777fe700 1 -- 172.21.15.79:0/3278377200 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926800e8e0 con 0x7f9260004850 2023-06-23T18:36:01.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.431+0000 7f92777fe700 1 -- 172.21.15.79:0/3278377200 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926800ec70 con 0x7f9260004850 2023-06-23T18:36:01.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7f7ea1e15700 1 -- 172.21.15.79:0/1536520635 >> v2:172.21.15.155:3300/0 conn(0x7f7e9c15ccc0 msgr2=0x7f7e9c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:01.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7f7ea1e15700 1 --2- 172.21.15.79:0/1536520635 >> v2:172.21.15.155:3300/0 conn(0x7f7e9c15ccc0 0x7f7e9c069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7f7ea1e15700 1 -- 172.21.15.79:0/1536520635 >> v2:172.21.15.79:3300/0 conn(0x7f7e9c0cbcd0 msgr2=0x7f7e9c0adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:01.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7f7ea1e15700 1 --2- 172.21.15.79:0/1536520635 >> v2:172.21.15.79:3300/0 conn(0x7f7e9c0cbcd0 0x7f7e9c0adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.430+0000 7f7ea1e15700 1 -- 172.21.15.79:0/1536520635 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c15c1e0 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.431+0000 7fd27df6b700 1 -- 172.21.15.79:0/876936772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd27804e240 con 0x7fd27806a3d0 2023-06-23T18:36:01.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.431+0000 7fd04870a700 1 -- 172.21.15.79:0/4157704538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd040146680 con 0x7fd04006b0b0 2023-06-23T18:36:01.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.431+0000 7fd04870a700 1 -- 172.21.15.79:0/4157704538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd040158d60 con 0x7fd04006b0b0 2023-06-23T18:36:01.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.431+0000 7f927f223700 1 -- 172.21.15.79:0/3278377200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260004850 msgr2=0x7f9260004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.431+0000 7f927f223700 1 --2- 172.21.15.79:0/3278377200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260004850 0x7f9260004cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9268009be0 tx=0x7f9260005880 comp rx=0 tx=0).stop 2023-06-23T18:36:01.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.431+0000 7f7ea1e15700 1 --2- 172.21.15.79:0/1536520635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e9c06a3d0 0x7f7e9c158a90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0098f0 tx=0x7f7e8c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f830d81cbcb9201a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.431+0000 7f95b8218700 1 -- 172.21.15.79:0/2593677289 shutdown_connections 2023-06-23T18:36:01.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.431+0000 7fd25ffff700 1 -- 172.21.15.79:0/876936772 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd2680037e0 con 0x7fd27806a3d0 2023-06-23T18:36:01.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.431+0000 7f95b8218700 1 -- 172.21.15.79:0/2593677289 wait complete. 2023-06-23T18:36:01.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.432+0000 7fc69ffff700 1 -- 172.21.15.79:0/2647280373 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a80052d0 con 0x7fc6b80706b0 2023-06-23T18:36:01.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.432+0000 7fc69ffff700 1 -- 172.21.15.79:0/2647280373 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a8005430 con 0x7fc6b80706b0 2023-06-23T18:36:01.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.432+0000 7f7e93fff700 1 -- 172.21.15.79:0/1536520635 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c011830 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.432+0000 7f7e93fff700 1 -- 172.21.15.79:0/1536520635 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c011990 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.432+0000 7f7e93fff700 1 -- 172.21.15.79:0/1536520635 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c011d20 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.432+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2647280373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b80b3890 con 0x7fc6b80706b0 2023-06-23T18:36:01.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.432+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2647280373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b80b3c20 con 0x7fc6b80706b0 2023-06-23T18:36:01.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.432+0000 7fc69ffff700 1 -- 172.21.15.79:0/2647280373 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a80038c0 con 0x7fc6b80706b0 2023-06-23T18:36:01.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.432+0000 7fd25ffff700 1 --2- 172.21.15.79:0/876936772 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd26006adf0 0x7fd26006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.432+0000 7fd25ffff700 1 -- 172.21.15.79:0/876936772 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd268088890 con 0x7fd27806a3d0 2023-06-23T18:36:01.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.433+0000 7f95b8218700 1 Processor -- start 2023-06-23T18:36:01.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.433+0000 7f95b8218700 1 -- start start 2023-06-23T18:36:01.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.433+0000 7fd276ffd700 1 --2- 172.21.15.79:0/876936772 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd26006adf0 0x7fd26006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.433+0000 7f7ea3878700 1 -- 172.21.15.79:0/1536520635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e9c06a3d0 msgr2=0x7f7e9c158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.433+0000 7f7ea3878700 1 --2- 172.21.15.79:0/1536520635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e9c06a3d0 0x7f7e9c158a90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0098f0 tx=0x7f7e8c009c00 comp rx=0 tx=0).stop 2023-06-23T18:36:01.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f7ea3878700 1 -- 172.21.15.79:0/1536520635 shutdown_connections 2023-06-23T18:36:01.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f7ea3878700 1 --2- 172.21.15.79:0/1536520635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e9c06a3d0 0x7f7e9c158a90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f7ea3878700 1 --2- 172.21.15.79:0/1536520635 >> v2:172.21.15.79:3300/0 conn(0x7f7e9c0cbcd0 0x7f7e9c0adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f7ea3878700 1 --2- 172.21.15.79:0/1536520635 >> v2:172.21.15.155:3300/0 conn(0x7f7e9c15ccc0 0x7f7e9c069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f7ea3878700 1 -- 172.21.15.79:0/1536520635 >> 172.21.15.79:0/1536520635 conn(0x7f7e9c0cb1c0 msgr2=0x7f7e9c0cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f95b8218700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95b00ce620 0x7f95b01fed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f95b8218700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95b0159170 0x7f95b00c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f95b8218700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95b00c5a80 0x7f95b015b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f95b8218700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b01ff240 con 0x7f95b00c5a80 2023-06-23T18:36:01.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f95b8218700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00b2ea0 con 0x7f95b0159170 2023-06-23T18:36:01.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f95b8218700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00b3180 con 0x7f95b00ce620 2023-06-23T18:36:01.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f95b5fb4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95b00ce620 0x7f95b01fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f95b5fb4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95b00ce620 0x7f95b01fed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:60534/0 (socket says 172.21.15.79:60534) 2023-06-23T18:36:01.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f95b5fb4700 1 -- 172.21.15.79:0/3344751837 learned_addr learned my addr 172.21.15.79:0/3344751837 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:01.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f95b5fb4700 1 -- 172.21.15.79:0/3344751837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95b0159170 msgr2=0x7f95b00c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f95b5fb4700 1 --2- 172.21.15.79:0/3344751837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95b0159170 0x7f95b00c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f95b5fb4700 1 -- 172.21.15.79:0/3344751837 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95b00c5a80 msgr2=0x7f95b015b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f95b5fb4700 1 --2- 172.21.15.79:0/3344751837 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95b00c5a80 0x7f95b015b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f95b5fb4700 1 -- 172.21.15.79:0/3344751837 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0094a0 con 0x7f95b00ce620 2023-06-23T18:36:01.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.435+0000 7f95b5fb4700 1 --2- 172.21.15.79:0/3344751837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95b00ce620 0x7f95b01fed00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0090b0 tx=0x7f95ac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.435+0000 7f95a67fc700 1 -- 172.21.15.79:0/3344751837 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac00fb70 con 0x7f95b00ce620 2023-06-23T18:36:01.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.433+0000 7fd04870a700 1 -- 172.21.15.79:0/4157704538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd040156f70 con 0x7fd04006b0b0 2023-06-23T18:36:01.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.435+0000 7f95a67fc700 1 -- 172.21.15.79:0/3344751837 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac00fcd0 con 0x7f95b00ce620 2023-06-23T18:36:01.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.435+0000 7f95a67fc700 1 -- 172.21.15.79:0/3344751837 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0106b0 con 0x7f95b00ce620 2023-06-23T18:36:01.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.435+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2647280373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6980052f0 con 0x7fc6b80706b0 2023-06-23T18:36:01.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.436+0000 7fd276ffd700 1 --2- 172.21.15.79:0/876936772 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd26006adf0 0x7fd26006d2a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd27816c7a0 tx=0x7fd26c00c630 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.436+0000 7fd036ffd700 1 -- 172.21.15.79:0/4157704538 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd03c003680 con 0x7fd04006b0b0 2023-06-23T18:36:01.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.436+0000 7fd036ffd700 1 --2- 172.21.15.79:0/4157704538 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd02406ace0 0x7fd02406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.436+0000 7fd036ffd700 1 -- 172.21.15.79:0/4157704538 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd03c088a90 con 0x7fd04006b0b0 2023-06-23T18:36:01.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.436+0000 7f95b8218700 1 -- 172.21.15.79:0/3344751837 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b00b3460 con 0x7f95b00ce620 2023-06-23T18:36:01.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.436+0000 7f95b8218700 1 -- 172.21.15.79:0/3344751837 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b00b3a20 con 0x7f95b00ce620 2023-06-23T18:36:01.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.437+0000 7f7ea3878700 1 -- 172.21.15.79:0/1536520635 shutdown_connections 2023-06-23T18:36:01.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.437+0000 7f95b8218700 1 -- 172.21.15.79:0/3344751837 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b00cf4b0 con 0x7f95b00ce620 2023-06-23T18:36:01.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.437+0000 7fd045ca5700 1 --2- 172.21.15.79:0/4157704538 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd02406ace0 0x7fd02406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.438+0000 7fd045ca5700 1 --2- 172.21.15.79:0/4157704538 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd02406ace0 0x7fd02406d190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd0400b3bb0 tx=0x7fd030008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.437+0000 7f7ea3878700 1 -- 172.21.15.79:0/1536520635 wait complete. 2023-06-23T18:36:01.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.439+0000 7f7ea3878700 1 Processor -- start 2023-06-23T18:36:01.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.439+0000 7f95a67fc700 1 -- 172.21.15.79:0/3344751837 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f95ac0233f0 con 0x7f95b00ce620 2023-06-23T18:36:01.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.433+0000 7f927f223700 1 -- 172.21.15.79:0/3278377200 shutdown_connections 2023-06-23T18:36:01.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.433+0000 7f927f223700 1 --2- 172.21.15.79:0/3278377200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260004850 0x7f9260004cc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.433+0000 7f927f223700 1 -- 172.21.15.79:0/3278377200 >> 172.21.15.79:0/3278377200 conn(0x7f92780cf3d0 msgr2=0x7f92780cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.433+0000 7f927f223700 1 -- 172.21.15.79:0/3278377200 shutdown_connections 2023-06-23T18:36:01.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.434+0000 7f927f223700 1 -- 172.21.15.79:0/3278377200 wait complete. 2023-06-23T18:36:01.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.436+0000 7f927f223700 1 Processor -- start 2023-06-23T18:36:01.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.436+0000 7f927f223700 1 -- start start 2023-06-23T18:36:01.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.437+0000 7f927f223700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260004850 0x7f9278206e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.437+0000 7f927f223700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92780ae300 0x7f92781eb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.437+0000 7f927f223700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278162300 0x7f9278162780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.437+0000 7f927f223700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9278162cc0 con 0x7f92780ae300 2023-06-23T18:36:01.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.437+0000 7f927f223700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92780af3e0 con 0x7f9260004850 2023-06-23T18:36:01.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.437+0000 7f927f223700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92782073c0 con 0x7f9278162300 2023-06-23T18:36:01.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.437+0000 7f927d7c0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278162300 0x7f9278162780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.437+0000 7f927d7c0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278162300 0x7f9278162780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:60542/0 (socket says 172.21.15.79:60542) 2023-06-23T18:36:01.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.437+0000 7f927d7c0700 1 -- 172.21.15.79:0/3039949989 learned_addr learned my addr 172.21.15.79:0/3039949989 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:01.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.437+0000 7f927cfbf700 1 --2- 172.21.15.79:0/3039949989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260004850 0x7f9278206e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.438+0000 7f927d7c0700 1 -- 172.21.15.79:0/3039949989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260004850 msgr2=0x7f9278206e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.438+0000 7f927d7c0700 1 --2- 172.21.15.79:0/3039949989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260004850 0x7f9278206e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.438+0000 7f927d7c0700 1 -- 172.21.15.79:0/3039949989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92780ae300 msgr2=0x7f92781eb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.438+0000 7f927d7c0700 1 --2- 172.21.15.79:0/3039949989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92780ae300 0x7f92781eb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.438+0000 7f927d7c0700 1 -- 172.21.15.79:0/3039949989 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92780ae880 con 0x7f9278162300 2023-06-23T18:36:01.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.438+0000 7f927d7c0700 1 --2- 172.21.15.79:0/3039949989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278162300 0x7f9278162780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f92780716e0 tx=0x7f926c00e7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.440+0000 7f95a67fc700 1 --2- 172.21.15.79:0/3344751837 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f959406ace0 0x7f959406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.440+0000 7f95a67fc700 1 -- 172.21.15.79:0/3344751837 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f95ac08b160 con 0x7f95b00ce620 2023-06-23T18:36:01.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.440+0000 7f95b57b3700 1 --2- 172.21.15.79:0/3344751837 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f959406ace0 0x7f959406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.440+0000 7f95b57b3700 1 --2- 172.21.15.79:0/3344751837 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f959406ace0 0x7f959406d190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f95b00b56c0 tx=0x7f95a8009250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.439+0000 7f7ea3878700 1 -- start start 2023-06-23T18:36:01.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.440+0000 7f7ea3878700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e9c06a3d0 0x7f7e9c0c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.440+0000 7f7ea3878700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e9c0cbcd0 0x7f7e9c0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.440+0000 7f7ea3878700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e9c15ccc0 0x7f7e9c15ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.440+0000 7f7ea3878700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0c8850 con 0x7f7e9c0cbcd0 2023-06-23T18:36:01.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.440+0000 7f7ea3878700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c146750 con 0x7f7e9c15ccc0 2023-06-23T18:36:01.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.440+0000 7f7ea3878700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0c7fb0 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.441+0000 7f7ea1614700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e9c06a3d0 0x7f7e9c0c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.441+0000 7f7ea1614700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e9c06a3d0 0x7f7e9c0c73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:60550/0 (socket says 172.21.15.79:60550) 2023-06-23T18:36:01.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.441+0000 7f7ea1614700 1 -- 172.21.15.79:0/2785169956 learned_addr learned my addr 172.21.15.79:0/2785169956 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:01.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.441+0000 7f7ea1e15700 1 --2- 172.21.15.79:0/2785169956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e9c15ccc0 0x7f7e9c15ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.441+0000 7f7ea0e13700 1 --2- 172.21.15.79:0/2785169956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e9c0cbcd0 0x7f7e9c0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.441+0000 7f7ea1614700 1 -- 172.21.15.79:0/2785169956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e9c15ccc0 msgr2=0x7f7e9c15ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.441+0000 7f7ea1614700 1 --2- 172.21.15.79:0/2785169956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e9c15ccc0 0x7f7e9c15ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.441+0000 7f7ea1614700 1 -- 172.21.15.79:0/2785169956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e9c0cbcd0 msgr2=0x7f7e9c0c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.441+0000 7f7ea1614700 1 --2- 172.21.15.79:0/2785169956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e9c0cbcd0 0x7f7e9c0c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.441+0000 7f7ea1614700 1 -- 172.21.15.79:0/2785169956 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c009550 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.441+0000 7f7ea1614700 1 --2- 172.21.15.79:0/2785169956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e9c06a3d0 0x7f7e9c0c73f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800e9b0 tx=0x7f7e9800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.442+0000 7fd25ffff700 1 -- 172.21.15.79:0/876936772 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd2680538a0 con 0x7fd27806a3d0 2023-06-23T18:36:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.442+0000 7f41bc83e700 1 Processor -- start 2023-06-23T18:36:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.442+0000 7fc69ffff700 1 -- 172.21.15.79:0/2647280373 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc6a8003a20 con 0x7fc6b80706b0 2023-06-23T18:36:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.442+0000 7f41bc83e700 1 -- start start 2023-06-23T18:36:01.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.442+0000 7fc69ffff700 1 --2- 172.21.15.79:0/2647280373 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc6a406ad30 0x7fc6a406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.443+0000 7fc6b77fe700 1 --2- 172.21.15.79:0/2647280373 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc6a406ad30 0x7fc6a406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.443+0000 7f92757fa700 1 -- 172.21.15.79:0/3039949989 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c00ee50 con 0x7f9278162300 2023-06-23T18:36:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.443+0000 7fc6b77fe700 1 --2- 172.21.15.79:0/2647280373 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc6a406ad30 0x7fc6a406d1e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac000f60 tx=0x7fc6ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.443+0000 7f41bc83e700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f41b815c4b0 0x7f41b80c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.443+0000 7f41bc83e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f41b80ca5e0 0x7f41b8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.443+0000 7f41bc83e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b80c91c0 con 0x7f41b80c3f40 2023-06-23T18:36:01.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.443+0000 7f41bc83e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b80cd350 con 0x7f41b80ca5e0 2023-06-23T18:36:01.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.443+0000 7f41bc83e700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b80c93f0 con 0x7f41b815c4b0 2023-06-23T18:36:01.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.443+0000 7f92757fa700 1 -- 172.21.15.79:0/3039949989 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926c0052e0 con 0x7f9278162300 2023-06-23T18:36:01.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.443+0000 7f927f223700 1 -- 172.21.15.79:0/3039949989 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92780aeb40 con 0x7f9278162300 2023-06-23T18:36:01.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.443+0000 7f92757fa700 1 -- 172.21.15.79:0/3039949989 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c003680 con 0x7f9278162300 2023-06-23T18:36:01.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.443+0000 7f927f223700 1 -- 172.21.15.79:0/3039949989 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92780af100 con 0x7f9278162300 2023-06-23T18:36:01.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.443+0000 7f41b6d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f41b80ca5e0 0x7f41b8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.443+0000 7f41b6d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f41b80ca5e0 0x7f41b8069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39556/0 (socket says 172.21.15.79:39556) 2023-06-23T18:36:01.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.444+0000 7f41b6d9d700 1 -- 172.21.15.79:0/2704571618 learned_addr learned my addr 172.21.15.79:0/2704571618 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.444+0000 7f41b659c700 1 --2- 172.21.15.79:0/2704571618 >> v2:172.21.15.155:3300/0 conn(0x7f41b815c4b0 0x7f41b80c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.444+0000 7f41b6d9d700 1 -- 172.21.15.79:0/2704571618 >> v2:172.21.15.155:3300/0 conn(0x7f41b815c4b0 msgr2=0x7f41b80c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.444+0000 7f41b6d9d700 1 --2- 172.21.15.79:0/2704571618 >> v2:172.21.15.155:3300/0 conn(0x7f41b815c4b0 0x7f41b80c3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.444+0000 7f41b6d9d700 1 -- 172.21.15.79:0/2704571618 >> v1:172.21.15.79:6789/0 conn(0x7f41b80c3f40 legacy=0x7f41b80c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.444+0000 7f41b6d9d700 1 -- 172.21.15.79:0/2704571618 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b80c96d0 con 0x7f41b80ca5e0 2023-06-23T18:36:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.444+0000 7f41b659c700 1 --2- 172.21.15.79:0/2704571618 >> v2:172.21.15.155:3300/0 conn(0x7f41b815c4b0 0x7f41b80c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.444+0000 7f41b6d9d700 1 --2- 172.21.15.79:0/2704571618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41b80ca5e0 0x7f41b8069e90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f41b80b4950 tx=0x7f41b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467b7b178fafea30 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.445+0000 7f927f223700 1 -- 172.21.15.79:0/3039949989 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92780cbe20 con 0x7f9278162300 2023-06-23T18:36:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.444+0000 7f7e91ffb700 1 -- 172.21.15.79:0/2785169956 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98009710 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.444+0000 7f7e91ffb700 1 -- 172.21.15.79:0/2785169956 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9800ee50 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.444+0000 7f7e91ffb700 1 -- 172.21.15.79:0/2785169956 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800ca50 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.444+0000 7f7ea3878700 1 -- 172.21.15.79:0/2785169956 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c0c8290 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.444+0000 7f7ea3878700 1 -- 172.21.15.79:0/2785169956 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c0c5980 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.446+0000 7f95a67fc700 1 -- 172.21.15.79:0/3344751837 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f95ac056420 con 0x7f95b00ce620 2023-06-23T18:36:01.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.446+0000 7f92757fa700 1 -- 172.21.15.79:0/3039949989 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f926c0037e0 con 0x7f9278162300 2023-06-23T18:36:01.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.446+0000 7f92757fa700 1 --2- 172.21.15.79:0/3039949989 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f925c06ad00 0x7f925c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.446+0000 7f92757fa700 1 -- 172.21.15.79:0/3039949989 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f926c088cb0 con 0x7f9278162300 2023-06-23T18:36:01.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.446+0000 7f927cfbf700 1 --2- 172.21.15.79:0/3039949989 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f925c06ad00 0x7f925c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.447+0000 7f927cfbf700 1 --2- 172.21.15.79:0/3039949989 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f925c06ad00 0x7f925c06d1b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9278063ac0 tx=0x7f927000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.446+0000 7f7ea3878700 1 -- 172.21.15.79:0/2785169956 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7c0052f0 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.446+0000 7f7e91ffb700 1 -- 172.21.15.79:0/2785169956 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e9800cbb0 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.447+0000 7f7e91ffb700 1 --2- 172.21.15.79:0/2785169956 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e8006adb0 0x7f7e8006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.447+0000 7f7e91ffb700 1 -- 172.21.15.79:0/2785169956 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7e98089370 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.447+0000 7f7ea0e13700 1 --2- 172.21.15.79:0/2785169956 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e8006adb0 0x7f7e8006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.447+0000 7fc69ffff700 1 -- 172.21.15.79:0/2647280373 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6a805bab0 con 0x7fc6b80706b0 2023-06-23T18:36:01.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.448+0000 7f7ea0e13700 1 --2- 172.21.15.79:0/2785169956 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e8006adb0 0x7f7e8006d260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c15d140 tx=0x7f7e9400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.448+0000 7f41b559a700 1 -- 172.21.15.79:0/2704571618 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b000e070 con 0x7f41b80ca5e0 2023-06-23T18:36:01.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.448+0000 7fd036ffd700 1 -- 172.21.15.79:0/4157704538 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd03c053b20 con 0x7fd04006b0b0 2023-06-23T18:36:01.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.450+0000 7f92757fa700 1 -- 172.21.15.79:0/3039949989 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f926c0578d0 con 0x7f9278162300 2023-06-23T18:36:01.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.454+0000 7f7e91ffb700 1 -- 172.21.15.79:0/2785169956 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7e98057f20 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.459+0000 7fc69ffff700 1 -- 172.21.15.79:0/2647280373 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc6a8053e30 con 0x7fc6b80706b0 2023-06-23T18:36:01.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.459+0000 7f41b559a700 1 -- 172.21.15.79:0/2704571618 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b000b9c0 con 0x7f41b80ca5e0 2023-06-23T18:36:01.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.459+0000 7f41b559a700 1 -- 172.21.15.79:0/2704571618 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b000bd70 con 0x7f41b80ca5e0 2023-06-23T18:36:01.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.460+0000 7f41bc83e700 1 -- 172.21.15.79:0/2704571618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41b80ca5e0 msgr2=0x7f41b8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.460+0000 7f41bc83e700 1 --2- 172.21.15.79:0/2704571618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41b80ca5e0 0x7f41b8069e90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f41b80b4950 tx=0x7f41b000b330 comp rx=0 tx=0).stop 2023-06-23T18:36:01.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.462+0000 7f41bc83e700 1 -- 172.21.15.79:0/2704571618 shutdown_connections 2023-06-23T18:36:01.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.462+0000 7f41bc83e700 1 --2- 172.21.15.79:0/2704571618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41b80ca5e0 0x7f41b8069e90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.462+0000 7f41bc83e700 1 --2- 172.21.15.79:0/2704571618 >> v2:172.21.15.155:3300/0 conn(0x7f41b815c4b0 0x7f41b80c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.462+0000 7f41bc83e700 1 -- 172.21.15.79:0/2704571618 >> 172.21.15.79:0/2704571618 conn(0x7f41b80cb1c0 msgr2=0x7f41b80cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.468+0000 7f41bc83e700 1 -- 172.21.15.79:0/2704571618 shutdown_connections 2023-06-23T18:36:01.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.468+0000 7f41bc83e700 1 -- 172.21.15.79:0/2704571618 wait complete. 2023-06-23T18:36:01.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.470+0000 7f41bc83e700 1 Processor -- start 2023-06-23T18:36:01.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.470+0000 7f41bc83e700 1 -- start start 2023-06-23T18:36:01.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.471+0000 7f41bc83e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41b80c3f40 0x7f41b80af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.471+0000 7f41bc83e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41b815c4b0 0x7f41b806ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.471+0000 7f41bc83e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41b806cfb0 0x7f41b82029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.471+0000 7f41bc83e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b8156bb0 con 0x7f41b815c4b0 2023-06-23T18:36:01.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.471+0000 7f41bc83e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b8156e50 con 0x7f41b80c3f40 2023-06-23T18:36:01.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.471+0000 7f41bc83e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b8157130 con 0x7f41b806cfb0 2023-06-23T18:36:01.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.472+0000 7f41b659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41b80c3f40 0x7f41b80af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.472+0000 7f41b6d9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41b806cfb0 0x7f41b82029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.473+0000 7f41b5d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41b815c4b0 0x7f41b806ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.473+0000 7f41b5d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41b815c4b0 0x7f41b806ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39568/0 (socket says 172.21.15.79:39568) 2023-06-23T18:36:01.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.473+0000 7f41b5d9b700 1 -- 172.21.15.79:0/3712456285 learned_addr learned my addr 172.21.15.79:0/3712456285 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:01.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.473+0000 7f41b5d9b700 1 -- 172.21.15.79:0/3712456285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41b806cfb0 msgr2=0x7f41b82029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.473+0000 7f41b5d9b700 1 --2- 172.21.15.79:0/3712456285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41b806cfb0 0x7f41b82029e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.473+0000 7f41b5d9b700 1 -- 172.21.15.79:0/3712456285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41b80c3f40 msgr2=0x7f41b80af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.473+0000 7f41b5d9b700 1 --2- 172.21.15.79:0/3712456285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41b80c3f40 0x7f41b80af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.473+0000 7f41b5d9b700 1 -- 172.21.15.79:0/3712456285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b0009b30 con 0x7f41b815c4b0 2023-06-23T18:36:01.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.473+0000 7f41b5d9b700 1 --2- 172.21.15.79:0/3712456285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41b815c4b0 0x7f41b806ca70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f41a800c4e0 tx=0x7f41a800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.474+0000 7f41a6ffd700 1 -- 172.21.15.79:0/3712456285 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a80052d0 con 0x7f41b815c4b0 2023-06-23T18:36:01.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.474+0000 7f41a6ffd700 1 -- 172.21.15.79:0/3712456285 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a8005430 con 0x7f41b815c4b0 2023-06-23T18:36:01.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.474+0000 7f41a6ffd700 1 -- 172.21.15.79:0/3712456285 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a8003880 con 0x7f41b815c4b0 2023-06-23T18:36:01.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.474+0000 7f41bc83e700 1 -- 172.21.15.79:0/3712456285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b8157410 con 0x7f41b815c4b0 2023-06-23T18:36:01.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.474+0000 7f41bc83e700 1 -- 172.21.15.79:0/3712456285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b806d4f0 con 0x7f41b815c4b0 2023-06-23T18:36:01.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.479+0000 7f41a6ffd700 1 -- 172.21.15.79:0/3712456285 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f41a80039e0 con 0x7f41b815c4b0 2023-06-23T18:36:01.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.479+0000 7f41a6ffd700 1 --2- 172.21.15.79:0/3712456285 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f419806adb0 0x7f419806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:01.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.479+0000 7f41a6ffd700 1 -- 172.21.15.79:0/3712456285 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f41a80888c0 con 0x7f41b815c4b0 2023-06-23T18:36:01.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.480+0000 7f41b659c700 1 --2- 172.21.15.79:0/3712456285 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f419806adb0 0x7f419806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:01.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.480+0000 7f41b659c700 1 --2- 172.21.15.79:0/3712456285 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f419806adb0 0x7f419806d260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009750 tx=0x7f41ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:01.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.486+0000 7f41bc83e700 1 -- 172.21.15.79:0/3712456285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419c0052f0 con 0x7f41b815c4b0 2023-06-23T18:36:01.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.496+0000 7f41a6ffd700 1 -- 172.21.15.79:0/3712456285 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f41a8053970 con 0x7f41b815c4b0 2023-06-23T18:36:01.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.781+0000 7f4add1cb700 1 -- 172.21.15.79:0/3754926951 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4ab40055a0 con 0x7f4ad815ccc0 2023-06-23T18:36:01.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.783+0000 7f4ac77fe700 1 -- 172.21.15.79:0/3754926951 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4ac802c020 con 0x7f4ad815ccc0 2023-06-23T18:36:01.790 INFO:teuthology.orchestra.run.smithi079.stdout:55834574854 2023-06-23T18:36:01.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.786+0000 7f4add1cb700 1 -- 172.21.15.79:0/3754926951 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4ac006af10 msgr2=0x7f4ac006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.786+0000 7f4add1cb700 1 --2- 172.21.15.79:0/3754926951 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4ac006af10 0x7f4ac006d3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4acc003d80 tx=0x7f4acc007a20 comp rx=0 tx=0).stop 2023-06-23T18:36:01.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.786+0000 7f4add1cb700 1 -- 172.21.15.79:0/3754926951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ad815ccc0 msgr2=0x7f4ad8147510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.786+0000 7f4add1cb700 1 --2- 172.21.15.79:0/3754926951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ad815ccc0 0x7f4ad8147510 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ad806fe10 tx=0x7f4ac8004750 comp rx=0 tx=0).stop 2023-06-23T18:36:01.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.786+0000 7f4add1cb700 1 -- 172.21.15.79:0/3754926951 shutdown_connections 2023-06-23T18:36:01.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.786+0000 7f4add1cb700 1 --2- 172.21.15.79:0/3754926951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ad815ccc0 0x7f4ad8147510 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.786+0000 7f4add1cb700 1 --2- 172.21.15.79:0/3754926951 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4ac006af10 0x7f4ac006d3c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.787+0000 7f4add1cb700 1 --2- 172.21.15.79:0/3754926951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad806a5a0 0x7f4ad806d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.787+0000 7f4add1cb700 1 --2- 172.21.15.79:0/3754926951 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ad80cbcd0 0x7f4ad8146fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.787+0000 7f4add1cb700 1 -- 172.21.15.79:0/3754926951 >> 172.21.15.79:0/3754926951 conn(0x7f4ad80cb1c0 msgr2=0x7f4ad80c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.787+0000 7f4add1cb700 1 -- 172.21.15.79:0/3754926951 shutdown_connections 2023-06-23T18:36:01.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.787+0000 7f4add1cb700 1 -- 172.21.15.79:0/3754926951 wait complete. 2023-06-23T18:36:01.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-06-23T18:36:01.811 DEBUG:teuthology.parallel:result is None 2023-06-23T18:36:01.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.824+0000 7f7ea3878700 1 -- 172.21.15.79:0/2785169956 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7e7c0055a0 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.825+0000 7f7e91ffb700 1 -- 172.21.15.79:0/2785169956 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7e9800ce60 con 0x7f7e9c06a3d0 2023-06-23T18:36:01.830 INFO:teuthology.orchestra.run.smithi079.stdout:55834574853 2023-06-23T18:36:01.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.827+0000 7f7ea3878700 1 -- 172.21.15.79:0/2785169956 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e8006adb0 msgr2=0x7f7e8006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.827+0000 7f7ea3878700 1 --2- 172.21.15.79:0/2785169956 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e8006adb0 0x7f7e8006d260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c15d140 tx=0x7f7e9400b040 comp rx=0 tx=0).stop 2023-06-23T18:36:01.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.827+0000 7f7ea3878700 1 -- 172.21.15.79:0/2785169956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e9c06a3d0 msgr2=0x7f7e9c0c73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.827+0000 7f7ea3878700 1 --2- 172.21.15.79:0/2785169956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e9c06a3d0 0x7f7e9c0c73f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800e9b0 tx=0x7f7e9800ecc0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.827+0000 7f7ea3878700 1 -- 172.21.15.79:0/2785169956 shutdown_connections 2023-06-23T18:36:01.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.827+0000 7f7ea3878700 1 --2- 172.21.15.79:0/2785169956 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e8006adb0 0x7f7e8006d260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.827+0000 7f7ea3878700 1 --2- 172.21.15.79:0/2785169956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e9c15ccc0 0x7f7e9c15ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.827+0000 7f7ea3878700 1 --2- 172.21.15.79:0/2785169956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e9c06a3d0 0x7f7e9c0c73f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.827+0000 7f7ea3878700 1 --2- 172.21.15.79:0/2785169956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e9c0cbcd0 0x7f7e9c0c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.827+0000 7f7ea3878700 1 -- 172.21.15.79:0/2785169956 >> 172.21.15.79:0/2785169956 conn(0x7f7e9c0cb1c0 msgr2=0x7f7e9c0cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.828+0000 7f7ea3878700 1 -- 172.21.15.79:0/2785169956 shutdown_connections 2023-06-23T18:36:01.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.828+0000 7f7ea3878700 1 -- 172.21.15.79:0/2785169956 wait complete. 2023-06-23T18:36:01.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2023-06-23T18:36:01.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.841+0000 7fd27df6b700 1 -- 172.21.15.79:0/876936772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd27806def0 con 0x7fd27806a3d0 2023-06-23T18:36:01.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.841+0000 7fd25ffff700 1 -- 172.21.15.79:0/876936772 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd268056ec0 con 0x7fd27806a3d0 2023-06-23T18:36:01.847 DEBUG:teuthology.parallel:result is None 2023-06-23T18:36:01.847 INFO:teuthology.orchestra.run.smithi079.stdout:55834574853 2023-06-23T18:36:01.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.844+0000 7fd27df6b700 1 -- 172.21.15.79:0/876936772 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd26006adf0 msgr2=0x7fd26006d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.844+0000 7fd27df6b700 1 --2- 172.21.15.79:0/876936772 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd26006adf0 0x7fd26006d2a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd27816c7a0 tx=0x7fd26c00c630 comp rx=0 tx=0).stop 2023-06-23T18:36:01.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.844+0000 7fd27df6b700 1 -- 172.21.15.79:0/876936772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd27806a3d0 msgr2=0x7fd27815e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.844+0000 7fd27df6b700 1 --2- 172.21.15.79:0/876936772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd27806a3d0 0x7fd27815e230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd26800c480 tx=0x7fd26800c790 comp rx=0 tx=0).stop 2023-06-23T18:36:01.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.844+0000 7fd27df6b700 1 -- 172.21.15.79:0/876936772 shutdown_connections 2023-06-23T18:36:01.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.844+0000 7fd27df6b700 1 --2- 172.21.15.79:0/876936772 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd26006adf0 0x7fd26006d2a0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd27816c7a0 tx=0x7fd26c00c630 comp rx=0 tx=0).stop 2023-06-23T18:36:01.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.844+0000 7fd27df6b700 1 --2- 172.21.15.79:0/876936772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2780add50 0x7fd278165d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.844+0000 7fd27df6b700 1 --2- 172.21.15.79:0/876936772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd27806a3d0 0x7fd27815e230 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.844+0000 7fd27df6b700 1 --2- 172.21.15.79:0/876936772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2780894e0 0x7fd278165850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.844+0000 7fd27df6b700 1 -- 172.21.15.79:0/876936772 >> 172.21.15.79:0/876936772 conn(0x7fd2780cb1c0 msgr2=0x7fd2780c9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.844+0000 7f95b8218700 1 -- 172.21.15.79:0/3344751837 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f95b00c5e90 con 0x7f95b00ce620 2023-06-23T18:36:01.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.845+0000 7fd27df6b700 1 -- 172.21.15.79:0/876936772 shutdown_connections 2023-06-23T18:36:01.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.845+0000 7fd27df6b700 1 -- 172.21.15.79:0/876936772 wait complete. 2023-06-23T18:36:01.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.845+0000 7f95a67fc700 1 -- 172.21.15.79:0/3344751837 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f95ac059a40 con 0x7f95b00ce620 2023-06-23T18:36:01.851 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-23T18:36:01.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.848+0000 7fd04870a700 1 -- 172.21.15.79:0/4157704538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd040146ba0 con 0x7fd04006b0b0 2023-06-23T18:36:01.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.848+0000 7fd036ffd700 1 -- 172.21.15.79:0/4157704538 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd03c057140 con 0x7fd04006b0b0 2023-06-23T18:36:01.853 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-23T18:36:01.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.849+0000 7f927f223700 1 -- 172.21.15.79:0/3039949989 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9278164e50 con 0x7f9278162300 2023-06-23T18:36:01.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.850+0000 7f92757fa700 1 -- 172.21.15.79:0/3039949989 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f926c019090 con 0x7f9278162300 2023-06-23T18:36:01.858 INFO:teuthology.orchestra.run.smithi079.stdout:55834574854 2023-06-23T18:36:01.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.850+0000 7f95b8218700 1 -- 172.21.15.79:0/3344751837 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f959406ace0 msgr2=0x7f959406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.850+0000 7f95b8218700 1 --2- 172.21.15.79:0/3344751837 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f959406ace0 0x7f959406d190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f95b00b56c0 tx=0x7f95a8009250 comp rx=0 tx=0).stop 2023-06-23T18:36:01.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.850+0000 7f95b8218700 1 -- 172.21.15.79:0/3344751837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95b00ce620 msgr2=0x7f95b01fed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.850+0000 7f95b8218700 1 --2- 172.21.15.79:0/3344751837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95b00ce620 0x7f95b01fed00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0090b0 tx=0x7f95ac004750 comp rx=0 tx=0).stop 2023-06-23T18:36:01.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.850+0000 7f95b8218700 1 -- 172.21.15.79:0/3344751837 shutdown_connections 2023-06-23T18:36:01.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.850+0000 7f95b8218700 1 --2- 172.21.15.79:0/3344751837 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95b00c5a80 0x7f95b015b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.850+0000 7f95b8218700 1 --2- 172.21.15.79:0/3344751837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95b00ce620 0x7f95b01fed00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.850+0000 7f95b8218700 1 --2- 172.21.15.79:0/3344751837 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f959406ace0 0x7f959406d190 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.850+0000 7f95b8218700 1 --2- 172.21.15.79:0/3344751837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95b0159170 0x7f95b00c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.850+0000 7f95b8218700 1 -- 172.21.15.79:0/3344751837 >> 172.21.15.79:0/3344751837 conn(0x7f95b006d660 msgr2=0x7f95b00af130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.850+0000 7f95b8218700 1 -- 172.21.15.79:0/3344751837 shutdown_connections 2023-06-23T18:36:01.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.850+0000 7f95b8218700 1 -- 172.21.15.79:0/3344751837 wait complete. 2023-06-23T18:36:01.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.851+0000 7fd04870a700 1 -- 172.21.15.79:0/4157704538 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd02406ace0 msgr2=0x7fd02406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.852+0000 7fd04870a700 1 --2- 172.21.15.79:0/4157704538 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd02406ace0 0x7fd02406d190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd0400b3bb0 tx=0x7fd030008030 comp rx=0 tx=0).stop 2023-06-23T18:36:01.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.852+0000 7fd04870a700 1 -- 172.21.15.79:0/4157704538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd04006b0b0 msgr2=0x7fd040158820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.852+0000 7fd04870a700 1 --2- 172.21.15.79:0/4157704538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd04006b0b0 0x7fd040158820 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00c440 tx=0x7fd03c00c750 comp rx=0 tx=0).stop 2023-06-23T18:36:01.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.852+0000 7fd04870a700 1 -- 172.21.15.79:0/4157704538 shutdown_connections 2023-06-23T18:36:01.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.852+0000 7fd04870a700 1 --2- 172.21.15.79:0/4157704538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0400cad80 0x7fd040144680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.852+0000 7fd04870a700 1 --2- 172.21.15.79:0/4157704538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd04006b0b0 0x7fd040158820 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.852+0000 7fd04870a700 1 --2- 172.21.15.79:0/4157704538 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd02406ace0 0x7fd02406d190 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd0400b3bb0 tx=0x7fd030008030 comp rx=0 tx=0).stop 2023-06-23T18:36:01.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.852+0000 7fd04870a700 1 --2- 172.21.15.79:0/4157704538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd040070b10 0x7fd040144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.852+0000 7fd04870a700 1 -- 172.21.15.79:0/4157704538 >> 172.21.15.79:0/4157704538 conn(0x7fd0400b17e0 msgr2=0x7fd0400c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.853+0000 7fd04870a700 1 -- 172.21.15.79:0/4157704538 shutdown_connections 2023-06-23T18:36:01.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.853+0000 7fd04870a700 1 -- 172.21.15.79:0/4157704538 wait complete. 2023-06-23T18:36:01.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.859+0000 7f927f223700 1 -- 172.21.15.79:0/3039949989 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f925c06ad00 msgr2=0x7f925c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.859+0000 7f927f223700 1 --2- 172.21.15.79:0/3039949989 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f925c06ad00 0x7f925c06d1b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9278063ac0 tx=0x7f927000b040 comp rx=0 tx=0).stop 2023-06-23T18:36:01.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.859+0000 7f927f223700 1 -- 172.21.15.79:0/3039949989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278162300 msgr2=0x7f9278162780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.859+0000 7f927f223700 1 --2- 172.21.15.79:0/3039949989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278162300 0x7f9278162780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f92780716e0 tx=0x7f926c00e7b0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.859+0000 7f927f223700 1 -- 172.21.15.79:0/3039949989 shutdown_connections 2023-06-23T18:36:01.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.859+0000 7f927f223700 1 --2- 172.21.15.79:0/3039949989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278162300 0x7f9278162780 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.859+0000 7f927f223700 1 --2- 172.21.15.79:0/3039949989 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f925c06ad00 0x7f925c06d1b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.859+0000 7f927f223700 1 --2- 172.21.15.79:0/3039949989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260004850 0x7f9278206e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.859+0000 7f927f223700 1 --2- 172.21.15.79:0/3039949989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92780ae300 0x7f92781eb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.859+0000 7f927f223700 1 -- 172.21.15.79:0/3039949989 >> 172.21.15.79:0/3039949989 conn(0x7f92780cf3d0 msgr2=0x7f927815bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.859+0000 7f927f223700 1 -- 172.21.15.79:0/3039949989 shutdown_connections 2023-06-23T18:36:01.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.859+0000 7f927f223700 1 -- 172.21.15.79:0/3039949989 wait complete. 2023-06-23T18:36:01.869 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2023-06-23T18:36:01.869 DEBUG:teuthology.parallel:result is None 2023-06-23T18:36:01.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.867+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2647280373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc698005870 con 0x7fc6b80706b0 2023-06-23T18:36:01.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.868+0000 7fc69ffff700 1 -- 172.21.15.79:0/2647280373 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc6a8017090 con 0x7fc6b80706b0 2023-06-23T18:36:01.873 INFO:teuthology.orchestra.run.smithi079.stdout:55834574854 2023-06-23T18:36:01.876 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-06-23T18:36:01.877 DEBUG:teuthology.parallel:result is None 2023-06-23T18:36:01.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-06-23T18:36:01.878 DEBUG:teuthology.parallel:result is None 2023-06-23T18:36:01.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.871+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2647280373 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc6a406ad30 msgr2=0x7fc6a406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.871+0000 7fc6bdc98700 1 --2- 172.21.15.79:0/2647280373 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc6a406ad30 0x7fc6a406d1e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac000f60 tx=0x7fc6ac009040 comp rx=0 tx=0).stop 2023-06-23T18:36:01.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.871+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2647280373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6b80706b0 msgr2=0x7fc6b80c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.871+0000 7fc6bdc98700 1 --2- 172.21.15.79:0/2647280373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6b80706b0 0x7fc6b80c8000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800c520 tx=0x7fc6a800c8e0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.871+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2647280373 shutdown_connections 2023-06-23T18:36:01.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.871+0000 7fc6bdc98700 1 --2- 172.21.15.79:0/2647280373 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6b8159130 0x7fc6b81fa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.871+0000 7fc6bdc98700 1 --2- 172.21.15.79:0/2647280373 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc6a406ad30 0x7fc6a406d1e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.871+0000 7fc6bdc98700 1 --2- 172.21.15.79:0/2647280373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6b8070230 0x7fc6b8153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.871+0000 7fc6bdc98700 1 --2- 172.21.15.79:0/2647280373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6b80706b0 0x7fc6b80c8000 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.871+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2647280373 >> 172.21.15.79:0/2647280373 conn(0x7fc6b80cf3d0 msgr2=0x7fc6b806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.872+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2647280373 shutdown_connections 2023-06-23T18:36:01.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.872+0000 7fc6bdc98700 1 -- 172.21.15.79:0/2647280373 wait complete. 2023-06-23T18:36:01.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-06-23T18:36:01.881 DEBUG:teuthology.parallel:result is None 2023-06-23T18:36:01.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.888+0000 7f41bc83e700 1 -- 172.21.15.79:0/3712456285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f419c0055a0 con 0x7f41b815c4b0 2023-06-23T18:36:01.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.888+0000 7f41a6ffd700 1 -- 172.21.15.79:0/3712456285 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f41a8017020 con 0x7f41b815c4b0 2023-06-23T18:36:01.893 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-23T18:36:01.894 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-06-23T18:36:01.894 DEBUG:teuthology.parallel:result is None 2023-06-23T18:36:01.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.890+0000 7f41bc83e700 1 -- 172.21.15.79:0/3712456285 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f419806adb0 msgr2=0x7f419806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.890+0000 7f41bc83e700 1 --2- 172.21.15.79:0/3712456285 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f419806adb0 0x7f419806d260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009750 tx=0x7f41ac008040 comp rx=0 tx=0).stop 2023-06-23T18:36:01.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.890+0000 7f41bc83e700 1 -- 172.21.15.79:0/3712456285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41b815c4b0 msgr2=0x7f41b806ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:01.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.890+0000 7f41bc83e700 1 --2- 172.21.15.79:0/3712456285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41b815c4b0 0x7f41b806ca70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f41a800c4e0 tx=0x7f41a800c8a0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.890+0000 7f41bc83e700 1 -- 172.21.15.79:0/3712456285 shutdown_connections 2023-06-23T18:36:01.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.890+0000 7f41bc83e700 1 --2- 172.21.15.79:0/3712456285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41b806cfb0 0x7f41b82029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.890+0000 7f41bc83e700 1 --2- 172.21.15.79:0/3712456285 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f419806adb0 0x7f419806d260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.890+0000 7f41bc83e700 1 --2- 172.21.15.79:0/3712456285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41b80c3f40 0x7f41b80af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.897 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.890+0000 7f41bc83e700 1 --2- 172.21.15.79:0/3712456285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41b815c4b0 0x7f41b806ca70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:01.897 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.890+0000 7f41bc83e700 1 -- 172.21.15.79:0/3712456285 >> 172.21.15.79:0/3712456285 conn(0x7f41b80cb1c0 msgr2=0x7f41b80ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:01.897 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.891+0000 7f41bc83e700 1 -- 172.21.15.79:0/3712456285 shutdown_connections 2023-06-23T18:36:01.897 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:01.891+0000 7f41bc83e700 1 -- 172.21.15.79:0/3712456285 wait complete. 2023-06-23T18:36:01.907 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-06-23T18:36:01.908 DEBUG:teuthology.parallel:result is None 2023-06-23T18:36:01.908 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-06-23T18:36:01.908 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:36:02.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.062+0000 7fc2be1c1700 1 Processor -- start 2023-06-23T18:36:02.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.063+0000 7fc2be1c1700 1 -- start start 2023-06-23T18:36:02.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.063+0000 7fc2be1c1700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2b815a380 0x7fc2b80ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.063+0000 7fc2be1c1700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc2b80cea30 0x7fc2b80b1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.063+0000 7fc2be1c1700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b80cee70 con 0x7fc2b815a750 2023-06-23T18:36:02.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.063+0000 7fc2be1c1700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b80b1d40 con 0x7fc2b80cea30 2023-06-23T18:36:02.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.063+0000 7fc2be1c1700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b80b2020 con 0x7fc2b815a380 2023-06-23T18:36:02.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.064+0000 7fc2b77fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2b815a380 0x7fc2b80ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.064+0000 7fc2b7fff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc2b80cea30 0x7fc2b80b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.064+0000 7fc2b6ffd700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fc2b815a750 0x7fc2b80ade50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:50884/0 (socket says 172.21.15.79:50884) 2023-06-23T18:36:02.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.064+0000 7fc2b6ffd700 1 -- 172.21.15.79:0/1995915374 learned_addr learned my addr 172.21.15.79:0/1995915374 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:02.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.064+0000 7fc2b67fc700 1 -- 172.21.15.79:0/1995915374 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933955813 0 0) 0x7fc2b80cee70 con 0x7fc2b815a750 2023-06-23T18:36:02.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.064+0000 7fc2b67fc700 1 -- 172.21.15.79:0/1995915374 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2a40035b0 con 0x7fc2b815a750 2023-06-23T18:36:02.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.064+0000 7fc2b7fff700 1 -- 172.21.15.79:0/1995915374 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2b815a380 msgr2=0x7fc2b80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:02.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.064+0000 7fc2b7fff700 1 --2- 172.21.15.79:0/1995915374 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2b815a380 0x7fc2b80ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.064+0000 7fc2b7fff700 1 -- 172.21.15.79:0/1995915374 >> v1:172.21.15.155:6789/0 conn(0x7fc2b815a750 legacy=0x7fc2b80ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.064+0000 7fc2b7fff700 1 -- 172.21.15.79:0/1995915374 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b80ad650 con 0x7fc2b80cea30 2023-06-23T18:36:02.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.064+0000 7fc2b7fff700 1 --2- 172.21.15.79:0/1995915374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b80cea30 0x7fc2b80b1010 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009d90 tx=0x7fc2b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26e0261270472406 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:02.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.064+0000 7fc2b67fc700 1 -- 172.21.15.79:0/1995915374 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b000e070 con 0x7fc2b80cea30 2023-06-23T18:36:02.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.064+0000 7fc2b67fc700 1 -- 172.21.15.79:0/1995915374 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b000bb20 con 0x7fc2b80cea30 2023-06-23T18:36:02.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.064+0000 7fc2b67fc700 1 -- 172.21.15.79:0/1995915374 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0012430 con 0x7fc2b80cea30 2023-06-23T18:36:02.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.065+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1995915374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b80cea30 msgr2=0x7fc2b80b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.065+0000 7fc2be1c1700 1 --2- 172.21.15.79:0/1995915374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b80cea30 0x7fc2b80b1010 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009d90 tx=0x7fc2b000b330 comp rx=0 tx=0).stop 2023-06-23T18:36:02.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.065+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1995915374 shutdown_connections 2023-06-23T18:36:02.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.065+0000 7fc2be1c1700 1 --2- 172.21.15.79:0/1995915374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b80cea30 0x7fc2b80b1010 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.065+0000 7fc2be1c1700 1 --2- 172.21.15.79:0/1995915374 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2b815a380 0x7fc2b80ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.065+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1995915374 >> 172.21.15.79:0/1995915374 conn(0x7fc2b8089fc0 msgr2=0x7fc2b8099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:02.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.065+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1995915374 shutdown_connections 2023-06-23T18:36:02.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.065+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1995915374 wait complete. 2023-06-23T18:36:02.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.066+0000 7fc2be1c1700 1 Processor -- start 2023-06-23T18:36:02.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.066+0000 7fc2be1c1700 1 -- start start 2023-06-23T18:36:02.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2be1c1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2b80cea30 0x7fc2b81e46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2be1c1700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2b815a380 0x7fc2b815c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2be1c1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b815a750 0x7fc2b814b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2be1c1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b815af80 con 0x7fc2b80cea30 2023-06-23T18:36:02.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2be1c1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b80c69c0 con 0x7fc2b815a750 2023-06-23T18:36:02.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2be1c1700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b80c6120 con 0x7fc2b815a380 2023-06-23T18:36:02.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2b77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2b80cea30 0x7fc2b81e46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2b77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2b80cea30 0x7fc2b81e46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39578/0 (socket says 172.21.15.79:39578) 2023-06-23T18:36:02.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2b77fe700 1 -- 172.21.15.79:0/1528274587 learned_addr learned my addr 172.21.15.79:0/1528274587 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:02.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2b7fff700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b815a750 0x7fc2b814b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2b6ffd700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2b815a380 0x7fc2b815c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2b77fe700 1 -- 172.21.15.79:0/1528274587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2b815a380 msgr2=0x7fc2b815c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2b77fe700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2b815a380 0x7fc2b815c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2b77fe700 1 -- 172.21.15.79:0/1528274587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b815a750 msgr2=0x7fc2b814b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2b77fe700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b815a750 0x7fc2b814b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.067+0000 7fc2b77fe700 1 -- 172.21.15.79:0/1528274587 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b0009a70 con 0x7fc2b80cea30 2023-06-23T18:36:02.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.068+0000 7fc2b7fff700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b815a750 0x7fc2b814b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:02.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.068+0000 7fc2b77fe700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2b80cea30 0x7fc2b81e46f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000e990 tx=0x7fc2a000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:02.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.068+0000 7fc2affff700 1 -- 172.21.15.79:0/1528274587 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a00096f0 con 0x7fc2b80cea30 2023-06-23T18:36:02.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.068+0000 7fc2affff700 1 -- 172.21.15.79:0/1528274587 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a000c850 con 0x7fc2b80cea30 2023-06-23T18:36:02.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.068+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1528274587 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b80c6400 con 0x7fc2b80cea30 2023-06-23T18:36:02.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.068+0000 7fc2affff700 1 -- 172.21.15.79:0/1528274587 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000cbc0 con 0x7fc2b80cea30 2023-06-23T18:36:02.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.068+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1528274587 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b815ac10 con 0x7fc2b80cea30 2023-06-23T18:36:02.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.069+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1528274587 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b815b260 con 0x7fc2b80cea30 2023-06-23T18:36:02.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.069+0000 7fc2affff700 1 -- 172.21.15.79:0/1528274587 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc2a000cd20 con 0x7fc2b80cea30 2023-06-23T18:36:02.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.070+0000 7fc2affff700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc29806ace0 0x7fc29806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.070+0000 7fc2affff700 1 -- 172.21.15.79:0/1528274587 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc2a0089920 con 0x7fc2b80cea30 2023-06-23T18:36:02.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.070+0000 7fc2b6ffd700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc29806ace0 0x7fc29806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.070+0000 7fc2b6ffd700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc29806ace0 0x7fc29806d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80c6d90 tx=0x7fc2a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:02.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.074+0000 7fc2affff700 1 -- 172.21.15.79:0/1528274587 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc2a0058460 con 0x7fc2b80cea30 2023-06-23T18:36:02.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.257+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1528274587 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc2b80ad930 con 0x7fc29806ace0 2023-06-23T18:36:02.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.265+0000 7fc2affff700 1 -- 172.21.15.79:0/1528274587 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286836 (secure 0 0 0) 0x7fc2a8005920 con 0x7fc29806ace0 2023-06-23T18:36:02.273 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:02.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.268+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1528274587 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc29806ace0 msgr2=0x7fc29806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.268+0000 7fc2be1c1700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc29806ace0 0x7fc29806d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80c6d90 tx=0x7fc2a8009040 comp rx=0 tx=0).stop 2023-06-23T18:36:02.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.268+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1528274587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2b80cea30 msgr2=0x7fc2b81e46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.268+0000 7fc2be1c1700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2b80cea30 0x7fc2b81e46f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000e990 tx=0x7fc2a000ed50 comp rx=0 tx=0).stop 2023-06-23T18:36:02.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.268+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1528274587 shutdown_connections 2023-06-23T18:36:02.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.268+0000 7fc2be1c1700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc29806ace0 0x7fc29806d190 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.268+0000 7fc2be1c1700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b815a750 0x7fc2b814b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.268+0000 7fc2be1c1700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2b80cea30 0x7fc2b81e46f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.268+0000 7fc2be1c1700 1 --2- 172.21.15.79:0/1528274587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2b815a380 0x7fc2b815c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.268+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1528274587 >> 172.21.15.79:0/1528274587 conn(0x7fc2b8089fc0 msgr2=0x7fc2b80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:02.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.268+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1528274587 shutdown_connections 2023-06-23T18:36:02.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.268+0000 7fc2be1c1700 1 -- 172.21.15.79:0/1528274587 wait complete. 2023-06-23T18:36:02.277 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-23T18:36:02.288 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-06-23T18:36:00.696780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218428,"kb_used_data":2812,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756292,"statfs":{"total":773094113280,"available":772870443008,"internally_reserved":0,"allocated":2879488,"data_stored":1776801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.585774"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287219+0000","last_change":"2023-06-23T18:35:53.534999+0000","last_active":"2023-06-23T18:35:55.287219+0000","last_peered":"2023-06-23T18:35:55.287219+0000","last_clean":"2023-06-23T18:35:55.287219+0000","last_became_active":"2023-06-23T18:35:53.534775+0000","last_became_peered":"2023-06-23T18:35:53.534775+0000","last_unstale":"2023-06-23T18:35:55.287219+0000","last_undegraded":"2023-06-23T18:35:55.287219+0000","last_fullsized":"2023-06-23T18:35:55.287219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:15:42.949792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527064+0000","last_change":"2023-06-23T18:35:53.537237+0000","last_active":"2023-06-23T18:35:54.527064+0000","last_peered":"2023-06-23T18:35:54.527064+0000","last_clean":"2023-06-23T18:35:54.527064+0000","last_became_active":"2023-06-23T18:35:53.536900+0000","last_became_peered":"2023-06-23T18:35:53.536900+0000","last_unstale":"2023-06-23T18:35:54.527064+0000","last_undegraded":"2023-06-23T18:35:54.527064+0000","last_fullsized":"2023-06-23T18:35:54.527064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:17.520121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526243+0000","last_change":"2023-06-23T18:35:53.527037+0000","last_active":"2023-06-23T18:35:54.526243+0000","last_peered":"2023-06-23T18:35:54.526243+0000","last_clean":"2023-06-23T18:35:54.526243+0000","last_became_active":"2023-06-23T18:35:53.526655+0000","last_became_peered":"2023-06-23T18:35:53.526655+0000","last_unstale":"2023-06-23T18:35:54.526243+0000","last_undegraded":"2023-06-23T18:35:54.526243+0000","last_fullsized":"2023-06-23T18:35:54.526243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:30.349972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257396+0000","last_change":"2023-06-23T18:35:53.540867+0000","last_active":"2023-06-23T18:35:55.257396+0000","last_peered":"2023-06-23T18:35:55.257396+0000","last_clean":"2023-06-23T18:35:55.257396+0000","last_became_active":"2023-06-23T18:35:53.540572+0000","last_became_peered":"2023-06-23T18:35:53.540572+0000","last_unstale":"2023-06-23T18:35:55.257396+0000","last_undegraded":"2023-06-23T18:35:55.257396+0000","last_fullsized":"2023-06-23T18:35:55.257396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:28.977806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287682+0000","last_change":"2023-06-23T18:35:53.535027+0000","last_active":"2023-06-23T18:35:55.287682+0000","last_peered":"2023-06-23T18:35:55.287682+0000","last_clean":"2023-06-23T18:35:55.287682+0000","last_became_active":"2023-06-23T18:35:53.534793+0000","last_became_peered":"2023-06-23T18:35:53.534793+0000","last_unstale":"2023-06-23T18:35:55.287682+0000","last_undegraded":"2023-06-23T18:35:55.287682+0000","last_fullsized":"2023-06-23T18:35:55.287682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:17.016870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526443+0000","last_change":"2023-06-23T18:35:53.524551+0000","last_active":"2023-06-23T18:35:54.526443+0000","last_peered":"2023-06-23T18:35:54.526443+0000","last_clean":"2023-06-23T18:35:54.526443+0000","last_became_active":"2023-06-23T18:35:53.524271+0000","last_became_peered":"2023-06-23T18:35:53.524271+0000","last_unstale":"2023-06-23T18:35:54.526443+0000","last_undegraded":"2023-06-23T18:35:54.526443+0000","last_fullsized":"2023-06-23T18:35:54.526443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:08:04.256786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793931+0000","last_change":"2023-06-23T18:35:53.540542+0000","last_active":"2023-06-23T18:35:54.793931+0000","last_peered":"2023-06-23T18:35:54.793931+0000","last_clean":"2023-06-23T18:35:54.793931+0000","last_became_active":"2023-06-23T18:35:53.539978+0000","last_became_peered":"2023-06-23T18:35:53.539978+0000","last_unstale":"2023-06-23T18:35:54.793931+0000","last_undegraded":"2023-06-23T18:35:54.793931+0000","last_fullsized":"2023-06-23T18:35:54.793931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:57:46.218466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525592+0000","last_change":"2023-06-23T18:35:53.533541+0000","last_active":"2023-06-23T18:35:54.525592+0000","last_peered":"2023-06-23T18:35:54.525592+0000","last_clean":"2023-06-23T18:35:54.525592+0000","last_became_active":"2023-06-23T18:35:53.531085+0000","last_became_peered":"2023-06-23T18:35:53.531085+0000","last_unstale":"2023-06-23T18:35:54.525592+0000","last_undegraded":"2023-06-23T18:35:54.525592+0000","last_fullsized":"2023-06-23T18:35:54.525592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:52.658179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288165+0000","last_change":"2023-06-23T18:35:53.534716+0000","last_active":"2023-06-23T18:35:55.288165+0000","last_peered":"2023-06-23T18:35:55.288165+0000","last_clean":"2023-06-23T18:35:55.288165+0000","last_became_active":"2023-06-23T18:35:53.534450+0000","last_became_peered":"2023-06-23T18:35:53.534450+0000","last_unstale":"2023-06-23T18:35:55.288165+0000","last_undegraded":"2023-06-23T18:35:55.288165+0000","last_fullsized":"2023-06-23T18:35:55.288165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527020+0000","last_change":"2023-06-23T18:35:53.533580+0000","last_active":"2023-06-23T18:35:54.527020+0000","last_peered":"2023-06-23T18:35:54.527020+0000","last_clean":"2023-06-23T18:35:54.527020+0000","last_became_active":"2023-06-23T18:35:53.533273+0000","last_became_peered":"2023-06-23T18:35:53.533273+0000","last_unstale":"2023-06-23T18:35:54.527020+0000","last_undegraded":"2023-06-23T18:35:54.527020+0000","last_fullsized":"2023-06-23T18:35:54.527020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526443+0000","last_change":"2023-06-23T18:35:53.537638+0000","last_active":"2023-06-23T18:35:54.526443+0000","last_peered":"2023-06-23T18:35:54.526443+0000","last_clean":"2023-06-23T18:35:54.526443+0000","last_became_active":"2023-06-23T18:35:53.537351+0000","last_became_peered":"2023-06-23T18:35:53.537351+0000","last_unstale":"2023-06-23T18:35:54.526443+0000","last_undegraded":"2023-06-23T18:35:54.526443+0000","last_fullsized":"2023-06-23T18:35:54.526443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:25:31.659636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288045+0000","last_change":"2023-06-23T18:35:53.530072+0000","last_active":"2023-06-23T18:35:55.288045+0000","last_peered":"2023-06-23T18:35:55.288045+0000","last_clean":"2023-06-23T18:35:55.288045+0000","last_became_active":"2023-06-23T18:35:53.529676+0000","last_became_peered":"2023-06-23T18:35:53.529676+0000","last_unstale":"2023-06-23T18:35:55.288045+0000","last_undegraded":"2023-06-23T18:35:55.288045+0000","last_fullsized":"2023-06-23T18:35:55.288045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:04:21.490439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287613+0000","last_change":"2023-06-23T18:35:53.535447+0000","last_active":"2023-06-23T18:35:55.287613+0000","last_peered":"2023-06-23T18:35:55.287613+0000","last_clean":"2023-06-23T18:35:55.287613+0000","last_became_active":"2023-06-23T18:35:53.535211+0000","last_became_peered":"2023-06-23T18:35:53.535211+0000","last_unstale":"2023-06-23T18:35:55.287613+0000","last_undegraded":"2023-06-23T18:35:55.287613+0000","last_fullsized":"2023-06-23T18:35:55.287613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:29:59.349736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526832+0000","last_change":"2023-06-23T18:35:53.530753+0000","last_active":"2023-06-23T18:35:54.526832+0000","last_peered":"2023-06-23T18:35:54.526832+0000","last_clean":"2023-06-23T18:35:54.526832+0000","last_became_active":"2023-06-23T18:35:53.530428+0000","last_became_peered":"2023-06-23T18:35:53.530428+0000","last_unstale":"2023-06-23T18:35:54.526832+0000","last_undegraded":"2023-06-23T18:35:54.526832+0000","last_fullsized":"2023-06-23T18:35:54.526832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:00.677969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794039+0000","last_change":"2023-06-23T18:35:53.540712+0000","last_active":"2023-06-23T18:35:54.794039+0000","last_peered":"2023-06-23T18:35:54.794039+0000","last_clean":"2023-06-23T18:35:54.794039+0000","last_became_active":"2023-06-23T18:35:53.540284+0000","last_became_peered":"2023-06-23T18:35:53.540284+0000","last_unstale":"2023-06-23T18:35:54.794039+0000","last_undegraded":"2023-06-23T18:35:54.794039+0000","last_fullsized":"2023-06-23T18:35:54.794039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:12.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525689+0000","last_change":"2023-06-23T18:35:53.524624+0000","last_active":"2023-06-23T18:35:54.525689+0000","last_peered":"2023-06-23T18:35:54.525689+0000","last_clean":"2023-06-23T18:35:54.525689+0000","last_became_active":"2023-06-23T18:35:53.524336+0000","last_became_peered":"2023-06-23T18:35:53.524336+0000","last_unstale":"2023-06-23T18:35:54.525689+0000","last_undegraded":"2023-06-23T18:35:54.525689+0000","last_fullsized":"2023-06-23T18:35:54.525689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:16.869108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288289+0000","last_change":"2023-06-23T18:35:53.534752+0000","last_active":"2023-06-23T18:35:55.288289+0000","last_peered":"2023-06-23T18:35:55.288289+0000","last_clean":"2023-06-23T18:35:55.288289+0000","last_became_active":"2023-06-23T18:35:53.534491+0000","last_became_peered":"2023-06-23T18:35:53.534491+0000","last_unstale":"2023-06-23T18:35:55.288289+0000","last_undegraded":"2023-06-23T18:35:55.288289+0000","last_fullsized":"2023-06-23T18:35:55.288289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:16.869108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526631+0000","last_change":"2023-06-23T18:35:53.530226+0000","last_active":"2023-06-23T18:35:54.526631+0000","last_peered":"2023-06-23T18:35:54.526631+0000","last_clean":"2023-06-23T18:35:54.526631+0000","last_became_active":"2023-06-23T18:35:53.529956+0000","last_became_peered":"2023-06-23T18:35:53.529956+0000","last_unstale":"2023-06-23T18:35:54.526631+0000","last_undegraded":"2023-06-23T18:35:54.526631+0000","last_fullsized":"2023-06-23T18:35:54.526631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:12.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287300+0000","last_change":"2023-06-23T18:35:53.540646+0000","last_active":"2023-06-23T18:35:55.287300+0000","last_peered":"2023-06-23T18:35:55.287300+0000","last_clean":"2023-06-23T18:35:55.287300+0000","last_became_active":"2023-06-23T18:35:53.540370+0000","last_became_peered":"2023-06-23T18:35:53.540370+0000","last_unstale":"2023-06-23T18:35:55.287300+0000","last_undegraded":"2023-06-23T18:35:55.287300+0000","last_fullsized":"2023-06-23T18:35:55.287300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:38:52.875168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526208+0000","last_change":"2023-06-23T18:35:53.539590+0000","last_active":"2023-06-23T18:35:54.526208+0000","last_peered":"2023-06-23T18:35:54.526208+0000","last_clean":"2023-06-23T18:35:54.526208+0000","last_became_active":"2023-06-23T18:35:53.539298+0000","last_became_peered":"2023-06-23T18:35:53.539298+0000","last_unstale":"2023-06-23T18:35:54.526208+0000","last_undegraded":"2023-06-23T18:35:54.526208+0000","last_fullsized":"2023-06-23T18:35:54.526208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:54:07.361405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794125+0000","last_change":"2023-06-23T18:35:53.543302+0000","last_active":"2023-06-23T18:35:54.794125+0000","last_peered":"2023-06-23T18:35:54.794125+0000","last_clean":"2023-06-23T18:35:54.794125+0000","last_became_active":"2023-06-23T18:35:53.542901+0000","last_became_peered":"2023-06-23T18:35:53.542901+0000","last_unstale":"2023-06-23T18:35:54.794125+0000","last_undegraded":"2023-06-23T18:35:54.794125+0000","last_fullsized":"2023-06-23T18:35:54.794125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:26:44.849536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287754+0000","last_change":"2023-06-23T18:35:53.525002+0000","last_active":"2023-06-23T18:35:55.287754+0000","last_peered":"2023-06-23T18:35:55.287754+0000","last_clean":"2023-06-23T18:35:55.287754+0000","last_became_active":"2023-06-23T18:35:53.524694+0000","last_became_peered":"2023-06-23T18:35:53.524694+0000","last_unstale":"2023-06-23T18:35:55.287754+0000","last_undegraded":"2023-06-23T18:35:55.287754+0000","last_fullsized":"2023-06-23T18:35:55.287754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:28.977806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288309+0000","last_change":"2023-06-23T18:35:53.532165+0000","last_active":"2023-06-23T18:35:55.288309+0000","last_peered":"2023-06-23T18:35:55.288309+0000","last_clean":"2023-06-23T18:35:55.288309+0000","last_became_active":"2023-06-23T18:35:53.531883+0000","last_became_peered":"2023-06-23T18:35:53.531883+0000","last_unstale":"2023-06-23T18:35:55.288309+0000","last_undegraded":"2023-06-23T18:35:55.288309+0000","last_fullsized":"2023-06-23T18:35:55.288309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793736+0000","last_change":"2023-06-23T18:35:53.543926+0000","last_active":"2023-06-23T18:35:54.793736+0000","last_peered":"2023-06-23T18:35:54.793736+0000","last_clean":"2023-06-23T18:35:54.793736+0000","last_became_active":"2023-06-23T18:35:53.543692+0000","last_became_peered":"2023-06-23T18:35:53.543692+0000","last_unstale":"2023-06-23T18:35:54.793736+0000","last_undegraded":"2023-06-23T18:35:54.793736+0000","last_fullsized":"2023-06-23T18:35:54.793736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:19:44.074507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527502+0000","last_change":"2023-06-23T18:35:53.539274+0000","last_active":"2023-06-23T18:35:54.527502+0000","last_peered":"2023-06-23T18:35:54.527502+0000","last_clean":"2023-06-23T18:35:54.527502+0000","last_became_active":"2023-06-23T18:35:53.539000+0000","last_became_peered":"2023-06-23T18:35:53.539000+0000","last_unstale":"2023-06-23T18:35:54.527502+0000","last_undegraded":"2023-06-23T18:35:54.527502+0000","last_fullsized":"2023-06-23T18:35:54.527502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:00.677969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527478+0000","last_change":"2023-06-23T18:35:53.532343+0000","last_active":"2023-06-23T18:35:54.527478+0000","last_peered":"2023-06-23T18:35:54.527478+0000","last_clean":"2023-06-23T18:35:54.527478+0000","last_became_active":"2023-06-23T18:35:53.532137+0000","last_became_peered":"2023-06-23T18:35:53.532137+0000","last_unstale":"2023-06-23T18:35:54.527478+0000","last_undegraded":"2023-06-23T18:35:54.527478+0000","last_fullsized":"2023-06-23T18:35:54.527478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:41.966301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526775+0000","last_change":"2023-06-23T18:35:53.541546+0000","last_active":"2023-06-23T18:35:54.526775+0000","last_peered":"2023-06-23T18:35:54.526775+0000","last_clean":"2023-06-23T18:35:54.526775+0000","last_became_active":"2023-06-23T18:35:53.541285+0000","last_became_peered":"2023-06-23T18:35:53.541285+0000","last_unstale":"2023-06-23T18:35:54.526775+0000","last_undegraded":"2023-06-23T18:35:54.526775+0000","last_fullsized":"2023-06-23T18:35:54.526775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:41.966301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527025+0000","last_change":"2023-06-23T18:35:53.130775+0000","last_active":"2023-06-23T18:35:54.527025+0000","last_peered":"2023-06-23T18:35:54.527025+0000","last_clean":"2023-06-23T18:35:54.527025+0000","last_became_active":"2023-06-23T18:35:53.130473+0000","last_became_peered":"2023-06-23T18:35:53.130473+0000","last_unstale":"2023-06-23T18:35:54.527025+0000","last_undegraded":"2023-06-23T18:35:54.527025+0000","last_fullsized":"2023-06-23T18:35:54.527025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:26:43.847769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288817+0000","last_change":"2023-06-23T18:35:53.530941+0000","last_active":"2023-06-23T18:35:55.288817+0000","last_peered":"2023-06-23T18:35:55.288817+0000","last_clean":"2023-06-23T18:35:55.288817+0000","last_became_active":"2023-06-23T18:35:53.530683+0000","last_became_peered":"2023-06-23T18:35:53.530683+0000","last_unstale":"2023-06-23T18:35:55.288817+0000","last_undegraded":"2023-06-23T18:35:55.288817+0000","last_fullsized":"2023-06-23T18:35:55.288817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:19:44.074507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527326+0000","last_change":"2023-06-23T18:35:53.132155+0000","last_active":"2023-06-23T18:35:54.527326+0000","last_peered":"2023-06-23T18:35:54.527326+0000","last_clean":"2023-06-23T18:35:54.527326+0000","last_became_active":"2023-06-23T18:35:53.131785+0000","last_became_peered":"2023-06-23T18:35:53.131785+0000","last_unstale":"2023-06-23T18:35:54.527326+0000","last_undegraded":"2023-06-23T18:35:54.527326+0000","last_fullsized":"2023-06-23T18:35:54.527326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:51.656412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288482+0000","last_change":"2023-06-23T18:35:53.537294+0000","last_active":"2023-06-23T18:35:55.288482+0000","last_peered":"2023-06-23T18:35:55.288482+0000","last_clean":"2023-06-23T18:35:55.288482+0000","last_became_active":"2023-06-23T18:35:53.536977+0000","last_became_peered":"2023-06-23T18:35:53.536977+0000","last_unstale":"2023-06-23T18:35:55.288482+0000","last_undegraded":"2023-06-23T18:35:55.288482+0000","last_fullsized":"2023-06-23T18:35:55.288482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:30.583311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288741+0000","last_change":"2023-06-23T18:35:53.135828+0000","last_active":"2023-06-23T18:35:55.288741+0000","last_peered":"2023-06-23T18:35:55.288741+0000","last_clean":"2023-06-23T18:35:55.288741+0000","last_became_active":"2023-06-23T18:35:53.135453+0000","last_became_peered":"2023-06-23T18:35:53.135453+0000","last_unstale":"2023-06-23T18:35:55.288741+0000","last_undegraded":"2023-06-23T18:35:55.288741+0000","last_fullsized":"2023-06-23T18:35:55.288741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:54:06.359638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257486+0000","last_change":"2023-06-23T18:35:53.535934+0000","last_active":"2023-06-23T18:35:55.257486+0000","last_peered":"2023-06-23T18:35:55.257486+0000","last_clean":"2023-06-23T18:35:55.257486+0000","last_became_active":"2023-06-23T18:35:53.535505+0000","last_became_peered":"2023-06-23T18:35:53.535505+0000","last_unstale":"2023-06-23T18:35:55.257486+0000","last_undegraded":"2023-06-23T18:35:55.257486+0000","last_fullsized":"2023-06-23T18:35:55.257486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288360+0000","last_change":"2023-06-23T18:35:53.130045+0000","last_active":"2023-06-23T18:35:55.288360+0000","last_peered":"2023-06-23T18:35:55.288360+0000","last_clean":"2023-06-23T18:35:55.288360+0000","last_became_active":"2023-06-23T18:35:53.129374+0000","last_became_peered":"2023-06-23T18:35:53.129374+0000","last_unstale":"2023-06-23T18:35:55.288360+0000","last_undegraded":"2023-06-23T18:35:55.288360+0000","last_fullsized":"2023-06-23T18:35:55.288360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:30.042205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288533+0000","last_change":"2023-06-23T18:35:53.540785+0000","last_active":"2023-06-23T18:35:55.288533+0000","last_peered":"2023-06-23T18:35:55.288533+0000","last_clean":"2023-06-23T18:35:55.288533+0000","last_became_active":"2023-06-23T18:35:53.540511+0000","last_became_peered":"2023-06-23T18:35:53.540511+0000","last_unstale":"2023-06-23T18:35:55.288533+0000","last_undegraded":"2023-06-23T18:35:55.288533+0000","last_fullsized":"2023-06-23T18:35:55.288533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:17.520121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526768+0000","last_change":"2023-06-23T18:35:53.133079+0000","last_active":"2023-06-23T18:35:54.526768+0000","last_peered":"2023-06-23T18:35:54.526768+0000","last_clean":"2023-06-23T18:35:54.526768+0000","last_became_active":"2023-06-23T18:35:53.132662+0000","last_became_peered":"2023-06-23T18:35:53.132662+0000","last_unstale":"2023-06-23T18:35:54.526768+0000","last_undegraded":"2023-06-23T18:35:54.526768+0000","last_fullsized":"2023-06-23T18:35:54.526768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:06.645556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288606+0000","last_change":"2023-06-23T18:35:53.541521+0000","last_active":"2023-06-23T18:35:55.288606+0000","last_peered":"2023-06-23T18:35:55.288606+0000","last_clean":"2023-06-23T18:35:55.288606+0000","last_became_active":"2023-06-23T18:35:53.541265+0000","last_became_peered":"2023-06-23T18:35:53.541265+0000","last_unstale":"2023-06-23T18:35:55.288606+0000","last_undegraded":"2023-06-23T18:35:55.288606+0000","last_fullsized":"2023-06-23T18:35:55.288606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:39.018447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793951+0000","last_change":"2023-06-23T18:35:53.131904+0000","last_active":"2023-06-23T18:35:54.793951+0000","last_peered":"2023-06-23T18:35:54.793951+0000","last_clean":"2023-06-23T18:35:54.793951+0000","last_became_active":"2023-06-23T18:35:53.131581+0000","last_became_peered":"2023-06-23T18:35:53.131581+0000","last_unstale":"2023-06-23T18:35:54.793951+0000","last_undegraded":"2023-06-23T18:35:54.793951+0000","last_fullsized":"2023-06-23T18:35:54.793951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:25:30.657869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527297+0000","last_change":"2023-06-23T18:35:53.542076+0000","last_active":"2023-06-23T18:35:54.527297+0000","last_peered":"2023-06-23T18:35:54.527297+0000","last_clean":"2023-06-23T18:35:54.527297+0000","last_became_active":"2023-06-23T18:35:53.541814+0000","last_became_peered":"2023-06-23T18:35:53.541814+0000","last_unstale":"2023-06-23T18:35:54.527297+0000","last_undegraded":"2023-06-23T18:35:54.527297+0000","last_fullsized":"2023-06-23T18:35:54.527297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:40:01.768196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526932+0000","last_change":"2023-06-23T18:35:53.131165+0000","last_active":"2023-06-23T18:35:54.526932+0000","last_peered":"2023-06-23T18:35:54.526932+0000","last_clean":"2023-06-23T18:35:54.526932+0000","last_became_active":"2023-06-23T18:35:53.130890+0000","last_became_peered":"2023-06-23T18:35:53.130890+0000","last_unstale":"2023-06-23T18:35:54.526932+0000","last_undegraded":"2023-06-23T18:35:54.526932+0000","last_fullsized":"2023-06-23T18:35:54.526932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:16:18.754209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526252+0000","last_change":"2023-06-23T18:35:53.541719+0000","last_active":"2023-06-23T18:35:54.526252+0000","last_peered":"2023-06-23T18:35:54.526252+0000","last_clean":"2023-06-23T18:35:54.526252+0000","last_became_active":"2023-06-23T18:35:53.541456+0000","last_became_peered":"2023-06-23T18:35:53.541456+0000","last_unstale":"2023-06-23T18:35:54.526252+0000","last_undegraded":"2023-06-23T18:35:54.526252+0000","last_fullsized":"2023-06-23T18:35:54.526252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:30:45.259696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794066+0000","last_change":"2023-06-23T18:35:53.136124+0000","last_active":"2023-06-23T18:35:54.794066+0000","last_peered":"2023-06-23T18:35:54.794066+0000","last_clean":"2023-06-23T18:35:54.794066+0000","last_became_active":"2023-06-23T18:35:53.135736+0000","last_became_peered":"2023-06-23T18:35:53.135736+0000","last_unstale":"2023-06-23T18:35:54.794066+0000","last_undegraded":"2023-06-23T18:35:54.794066+0000","last_fullsized":"2023-06-23T18:35:54.794066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:45:16.668072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.289031+0000","last_change":"2023-06-23T18:35:53.540586+0000","last_active":"2023-06-23T18:35:55.289031+0000","last_peered":"2023-06-23T18:35:55.289031+0000","last_clean":"2023-06-23T18:35:55.289031+0000","last_became_active":"2023-06-23T18:35:53.540355+0000","last_became_peered":"2023-06-23T18:35:53.540355+0000","last_unstale":"2023-06-23T18:35:55.289031+0000","last_undegraded":"2023-06-23T18:35:55.289031+0000","last_fullsized":"2023-06-23T18:35:55.289031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:02:35.233797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287933+0000","last_change":"2023-06-23T18:35:53.133113+0000","last_active":"2023-06-23T18:35:55.287933+0000","last_peered":"2023-06-23T18:35:55.287933+0000","last_clean":"2023-06-23T18:35:55.287933+0000","last_became_active":"2023-06-23T18:35:53.132833+0000","last_became_peered":"2023-06-23T18:35:53.132833+0000","last_unstale":"2023-06-23T18:35:55.287933+0000","last_undegraded":"2023-06-23T18:35:55.287933+0000","last_fullsized":"2023-06-23T18:35:55.287933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:46:59.676202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527016+0000","last_change":"2023-06-23T18:35:53.538309+0000","last_active":"2023-06-23T18:35:54.527016+0000","last_peered":"2023-06-23T18:35:54.527016+0000","last_clean":"2023-06-23T18:35:54.527016+0000","last_became_active":"2023-06-23T18:35:53.538037+0000","last_became_peered":"2023-06-23T18:35:53.538037+0000","last_unstale":"2023-06-23T18:35:54.527016+0000","last_undegraded":"2023-06-23T18:35:54.527016+0000","last_fullsized":"2023-06-23T18:35:54.527016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793844+0000","last_change":"2023-06-23T18:35:53.131958+0000","last_active":"2023-06-23T18:35:54.793844+0000","last_peered":"2023-06-23T18:35:54.793844+0000","last_clean":"2023-06-23T18:35:54.793844+0000","last_became_active":"2023-06-23T18:35:53.131643+0000","last_became_peered":"2023-06-23T18:35:53.131643+0000","last_unstale":"2023-06-23T18:35:54.793844+0000","last_undegraded":"2023-06-23T18:35:54.793844+0000","last_fullsized":"2023-06-23T18:35:54.793844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:57.730157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288109+0000","last_change":"2023-06-23T18:35:53.533318+0000","last_active":"2023-06-23T18:35:55.288109+0000","last_peered":"2023-06-23T18:35:55.288109+0000","last_clean":"2023-06-23T18:35:55.288109+0000","last_became_active":"2023-06-23T18:35:53.533081+0000","last_became_peered":"2023-06-23T18:35:53.533081+0000","last_unstale":"2023-06-23T18:35:55.288109+0000","last_undegraded":"2023-06-23T18:35:55.288109+0000","last_fullsized":"2023-06-23T18:35:55.288109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257090+0000","last_change":"2023-06-23T18:35:53.135982+0000","last_active":"2023-06-23T18:35:55.257090+0000","last_peered":"2023-06-23T18:35:55.257090+0000","last_clean":"2023-06-23T18:35:55.257090+0000","last_became_active":"2023-06-23T18:35:53.135591+0000","last_became_peered":"2023-06-23T18:35:53.135591+0000","last_unstale":"2023-06-23T18:35:55.257090+0000","last_undegraded":"2023-06-23T18:35:55.257090+0000","last_fullsized":"2023-06-23T18:35:55.257090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:40:00.766429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288988+0000","last_change":"2023-06-23T18:35:53.541883+0000","last_active":"2023-06-23T18:35:55.288988+0000","last_peered":"2023-06-23T18:35:55.288988+0000","last_clean":"2023-06-23T18:35:55.288988+0000","last_became_active":"2023-06-23T18:35:53.541630+0000","last_became_peered":"2023-06-23T18:35:53.541630+0000","last_unstale":"2023-06-23T18:35:55.288988+0000","last_undegraded":"2023-06-23T18:35:55.288988+0000","last_fullsized":"2023-06-23T18:35:55.288988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:08:04.256786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793955+0000","last_change":"2023-06-23T18:35:53.132583+0000","last_active":"2023-06-23T18:35:54.793955+0000","last_peered":"2023-06-23T18:35:54.793955+0000","last_clean":"2023-06-23T18:35:54.793955+0000","last_became_active":"2023-06-23T18:35:53.132142+0000","last_became_peered":"2023-06-23T18:35:53.132142+0000","last_unstale":"2023-06-23T18:35:54.793955+0000","last_undegraded":"2023-06-23T18:35:54.793955+0000","last_fullsized":"2023-06-23T18:35:54.793955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:02:34.232029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526009+0000","last_change":"2023-06-23T18:35:53.537549+0000","last_active":"2023-06-23T18:35:54.526009+0000","last_peered":"2023-06-23T18:35:54.526009+0000","last_clean":"2023-06-23T18:35:54.526009+0000","last_became_active":"2023-06-23T18:35:53.537281+0000","last_became_peered":"2023-06-23T18:35:53.537281+0000","last_unstale":"2023-06-23T18:35:54.526009+0000","last_undegraded":"2023-06-23T18:35:54.526009+0000","last_fullsized":"2023-06-23T18:35:54.526009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:17.016870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794202+0000","last_change":"2023-06-23T18:35:53.131954+0000","last_active":"2023-06-23T18:35:54.794202+0000","last_peered":"2023-06-23T18:35:54.794202+0000","last_clean":"2023-06-23T18:35:54.794202+0000","last_became_active":"2023-06-23T18:35:53.131624+0000","last_became_peered":"2023-06-23T18:35:53.131624+0000","last_unstale":"2023-06-23T18:35:54.794202+0000","last_undegraded":"2023-06-23T18:35:54.794202+0000","last_fullsized":"2023-06-23T18:35:54.794202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:16:18.754209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525847+0000","last_change":"2023-06-23T18:35:53.535467+0000","last_active":"2023-06-23T18:35:54.525847+0000","last_peered":"2023-06-23T18:35:54.525847+0000","last_clean":"2023-06-23T18:35:54.525847+0000","last_became_active":"2023-06-23T18:35:53.535187+0000","last_became_peered":"2023-06-23T18:35:53.535187+0000","last_unstale":"2023-06-23T18:35:54.525847+0000","last_undegraded":"2023-06-23T18:35:54.525847+0000","last_fullsized":"2023-06-23T18:35:54.525847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:31.043972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793955+0000","last_change":"2023-06-23T18:35:53.132445+0000","last_active":"2023-06-23T18:35:54.793955+0000","last_peered":"2023-06-23T18:35:54.793955+0000","last_clean":"2023-06-23T18:35:54.793955+0000","last_became_active":"2023-06-23T18:35:53.132108+0000","last_became_peered":"2023-06-23T18:35:53.132108+0000","last_unstale":"2023-06-23T18:35:54.793955+0000","last_undegraded":"2023-06-23T18:35:54.793955+0000","last_fullsized":"2023-06-23T18:35:54.793955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:44:02.521500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257388+0000","last_change":"2023-06-23T18:35:53.539086+0000","last_active":"2023-06-23T18:35:55.257388+0000","last_peered":"2023-06-23T18:35:55.257388+0000","last_clean":"2023-06-23T18:35:55.257388+0000","last_became_active":"2023-06-23T18:35:53.538755+0000","last_became_peered":"2023-06-23T18:35:53.538755+0000","last_unstale":"2023-06-23T18:35:55.257388+0000","last_undegraded":"2023-06-23T18:35:55.257388+0000","last_fullsized":"2023-06-23T18:35:55.257388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:07.647324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527056+0000","last_change":"2023-06-23T18:35:53.128100+0000","last_active":"2023-06-23T18:35:54.527056+0000","last_peered":"2023-06-23T18:35:54.527056+0000","last_clean":"2023-06-23T18:35:54.527056+0000","last_became_active":"2023-06-23T18:35:53.127782+0000","last_became_peered":"2023-06-23T18:35:53.127782+0000","last_unstale":"2023-06-23T18:35:54.527056+0000","last_undegraded":"2023-06-23T18:35:54.527056+0000","last_fullsized":"2023-06-23T18:35:54.527056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:15.867340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287463+0000","last_change":"2023-06-23T18:35:53.524781+0000","last_active":"2023-06-23T18:35:55.287463+0000","last_peered":"2023-06-23T18:35:55.287463+0000","last_clean":"2023-06-23T18:35:55.287463+0000","last_became_active":"2023-06-23T18:35:53.524526+0000","last_became_peered":"2023-06-23T18:35:53.524526+0000","last_unstale":"2023-06-23T18:35:55.287463+0000","last_undegraded":"2023-06-23T18:35:55.287463+0000","last_fullsized":"2023-06-23T18:35:55.287463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:40:01.768196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288801+0000","last_change":"2023-06-23T18:35:53.133550+0000","last_active":"2023-06-23T18:35:55.288801+0000","last_peered":"2023-06-23T18:35:55.288801+0000","last_clean":"2023-06-23T18:35:55.288801+0000","last_became_active":"2023-06-23T18:35:53.133287+0000","last_became_peered":"2023-06-23T18:35:53.133287+0000","last_unstale":"2023-06-23T18:35:55.288801+0000","last_undegraded":"2023-06-23T18:35:55.288801+0000","last_fullsized":"2023-06-23T18:35:55.288801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:30:44.257929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526733+0000","last_change":"2023-06-23T18:35:53.534074+0000","last_active":"2023-06-23T18:35:54.526733+0000","last_peered":"2023-06-23T18:35:54.526733+0000","last_clean":"2023-06-23T18:35:54.526733+0000","last_became_active":"2023-06-23T18:35:53.533789+0000","last_became_peered":"2023-06-23T18:35:53.533789+0000","last_unstale":"2023-06-23T18:35:54.526733+0000","last_undegraded":"2023-06-23T18:35:54.526733+0000","last_fullsized":"2023-06-23T18:35:54.526733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:58.731924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257244+0000","last_change":"2023-06-23T18:35:53.126156+0000","last_active":"2023-06-23T18:35:55.257244+0000","last_peered":"2023-06-23T18:35:55.257244+0000","last_clean":"2023-06-23T18:35:55.257244+0000","last_became_active":"2023-06-23T18:35:53.125640+0000","last_became_peered":"2023-06-23T18:35:53.125640+0000","last_unstale":"2023-06-23T18:35:55.257244+0000","last_undegraded":"2023-06-23T18:35:55.257244+0000","last_fullsized":"2023-06-23T18:35:55.257244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:16.015103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526631+0000","last_change":"2023-06-23T18:35:53.529926+0000","last_active":"2023-06-23T18:35:54.526631+0000","last_peered":"2023-06-23T18:35:54.526631+0000","last_clean":"2023-06-23T18:35:54.526631+0000","last_became_active":"2023-06-23T18:35:53.529619+0000","last_became_peered":"2023-06-23T18:35:53.529619+0000","last_unstale":"2023-06-23T18:35:54.526631+0000","last_undegraded":"2023-06-23T18:35:54.526631+0000","last_fullsized":"2023-06-23T18:35:54.526631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:44:03.523267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526406+0000","last_change":"2023-06-23T18:35:53.134550+0000","last_active":"2023-06-23T18:35:54.526406+0000","last_peered":"2023-06-23T18:35:54.526406+0000","last_clean":"2023-06-23T18:35:54.526406+0000","last_became_active":"2023-06-23T18:35:53.132753+0000","last_became_peered":"2023-06-23T18:35:53.132753+0000","last_unstale":"2023-06-23T18:35:54.526406+0000","last_undegraded":"2023-06-23T18:35:54.526406+0000","last_fullsized":"2023-06-23T18:35:54.526406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:19:43.072740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526469+0000","last_change":"2023-06-23T18:35:53.541907+0000","last_active":"2023-06-23T18:35:54.526469+0000","last_peered":"2023-06-23T18:35:54.526469+0000","last_clean":"2023-06-23T18:35:54.526469+0000","last_became_active":"2023-06-23T18:35:53.541545+0000","last_became_peered":"2023-06-23T18:35:53.541545+0000","last_unstale":"2023-06-23T18:35:54.526469+0000","last_undegraded":"2023-06-23T18:35:54.526469+0000","last_fullsized":"2023-06-23T18:35:54.526469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:02:35.233797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288368+0000","last_change":"2023-06-23T18:35:53.130943+0000","last_active":"2023-06-23T18:35:55.288368+0000","last_peered":"2023-06-23T18:35:55.288368+0000","last_clean":"2023-06-23T18:35:55.288368+0000","last_became_active":"2023-06-23T18:35:53.130544+0000","last_became_peered":"2023-06-23T18:35:53.130544+0000","last_unstale":"2023-06-23T18:35:55.288368+0000","last_undegraded":"2023-06-23T18:35:55.288368+0000","last_fullsized":"2023-06-23T18:35:55.288368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:18.373567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527635+0000","last_change":"2023-06-23T18:35:53.535355+0000","last_active":"2023-06-23T18:35:54.527635+0000","last_peered":"2023-06-23T18:35:54.527635+0000","last_clean":"2023-06-23T18:35:54.527635+0000","last_became_active":"2023-06-23T18:35:53.535087+0000","last_became_peered":"2023-06-23T18:35:53.535087+0000","last_unstale":"2023-06-23T18:35:54.527635+0000","last_undegraded":"2023-06-23T18:35:54.527635+0000","last_fullsized":"2023-06-23T18:35:54.527635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288978+0000","last_change":"2023-06-23T18:35:53.133469+0000","last_active":"2023-06-23T18:35:55.288978+0000","last_peered":"2023-06-23T18:35:55.288978+0000","last_clean":"2023-06-23T18:35:55.288978+0000","last_became_active":"2023-06-23T18:35:53.133211+0000","last_became_peered":"2023-06-23T18:35:53.133211+0000","last_unstale":"2023-06-23T18:35:55.288978+0000","last_undegraded":"2023-06-23T18:35:55.288978+0000","last_fullsized":"2023-06-23T18:35:55.288978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:57:45.216699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526357+0000","last_change":"2023-06-23T18:35:48.920009+0000","last_active":"2023-06-23T18:35:54.526357+0000","last_peered":"2023-06-23T18:35:54.526357+0000","last_clean":"2023-06-23T18:35:54.526357+0000","last_became_active":"2023-06-23T18:35:48.919686+0000","last_became_peered":"2023-06-23T18:35:48.919686+0000","last_unstale":"2023-06-23T18:35:54.526357+0000","last_undegraded":"2023-06-23T18:35:54.526357+0000","last_fullsized":"2023-06-23T18:35:54.526357+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:57:40.062937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288250+0000","last_change":"2023-06-23T18:35:53.538294+0000","last_active":"2023-06-23T18:35:55.288250+0000","last_peered":"2023-06-23T18:35:55.288250+0000","last_clean":"2023-06-23T18:35:55.288250+0000","last_became_active":"2023-06-23T18:35:53.537996+0000","last_became_peered":"2023-06-23T18:35:53.537996+0000","last_unstale":"2023-06-23T18:35:55.288250+0000","last_undegraded":"2023-06-23T18:35:55.288250+0000","last_fullsized":"2023-06-23T18:35:55.288250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288977+0000","last_change":"2023-06-23T18:35:47.962164+0000","last_active":"2023-06-23T18:35:55.288977+0000","last_peered":"2023-06-23T18:35:55.288977+0000","last_clean":"2023-06-23T18:35:55.288977+0000","last_became_active":"2023-06-23T18:35:47.961790+0000","last_became_peered":"2023-06-23T18:35:47.961790+0000","last_unstale":"2023-06-23T18:35:55.288977+0000","last_undegraded":"2023-06-23T18:35:55.288977+0000","last_fullsized":"2023-06-23T18:35:55.288977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:25:25.504108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.944056+0000","last_change":"2023-06-23T18:35:53.126850+0000","last_active":"2023-06-23T18:35:55.944056+0000","last_peered":"2023-06-23T18:35:55.944056+0000","last_clean":"2023-06-23T18:35:55.944056+0000","last_became_active":"2023-06-23T18:35:53.126560+0000","last_became_peered":"2023-06-23T18:35:53.126560+0000","last_unstale":"2023-06-23T18:35:55.944056+0000","last_undegraded":"2023-06-23T18:35:55.944056+0000","last_fullsized":"2023-06-23T18:35:55.944056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:38.016680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526607+0000","last_change":"2023-06-23T18:35:53.531624+0000","last_active":"2023-06-23T18:35:54.526607+0000","last_peered":"2023-06-23T18:35:54.526607+0000","last_clean":"2023-06-23T18:35:54.526607+0000","last_became_active":"2023-06-23T18:35:53.531197+0000","last_became_peered":"2023-06-23T18:35:53.531197+0000","last_unstale":"2023-06-23T18:35:54.526607+0000","last_undegraded":"2023-06-23T18:35:54.526607+0000","last_fullsized":"2023-06-23T18:35:54.526607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:17.520121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527435+0000","last_change":"2023-06-23T18:35:47.965867+0000","last_active":"2023-06-23T18:35:54.527435+0000","last_peered":"2023-06-23T18:35:54.527435+0000","last_clean":"2023-06-23T18:35:54.527435+0000","last_became_active":"2023-06-23T18:35:47.965580+0000","last_became_peered":"2023-06-23T18:35:47.965580+0000","last_unstale":"2023-06-23T18:35:54.527435+0000","last_undegraded":"2023-06-23T18:35:54.527435+0000","last_fullsized":"2023-06-23T18:35:54.527435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:01.491795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793869+0000","last_change":"2023-06-23T18:35:47.968534+0000","last_active":"2023-06-23T18:35:54.793869+0000","last_peered":"2023-06-23T18:35:54.793869+0000","last_clean":"2023-06-23T18:35:54.793869+0000","last_became_active":"2023-06-23T18:35:47.967508+0000","last_became_peered":"2023-06-23T18:35:47.967508+0000","last_unstale":"2023-06-23T18:35:54.793869+0000","last_undegraded":"2023-06-23T18:35:54.793869+0000","last_fullsized":"2023-06-23T18:35:54.793869+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:07:58.101257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287523+0000","last_change":"2023-06-23T18:35:53.126632+0000","last_active":"2023-06-23T18:35:55.287523+0000","last_peered":"2023-06-23T18:35:55.287523+0000","last_clean":"2023-06-23T18:35:55.287523+0000","last_became_active":"2023-06-23T18:35:53.126336+0000","last_became_peered":"2023-06-23T18:35:53.126336+0000","last_unstale":"2023-06-23T18:35:55.287523+0000","last_undegraded":"2023-06-23T18:35:55.287523+0000","last_fullsized":"2023-06-23T18:35:55.287523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:51.656412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288240+0000","last_change":"2023-06-23T18:35:53.524885+0000","last_active":"2023-06-23T18:35:55.288240+0000","last_peered":"2023-06-23T18:35:55.288240+0000","last_clean":"2023-06-23T18:35:55.288240+0000","last_became_active":"2023-06-23T18:35:53.524630+0000","last_became_peered":"2023-06-23T18:35:53.524630+0000","last_unstale":"2023-06-23T18:35:55.288240+0000","last_undegraded":"2023-06-23T18:35:55.288240+0000","last_fullsized":"2023-06-23T18:35:55.288240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793589+0000","last_change":"2023-06-23T18:35:48.736407+0000","last_active":"2023-06-23T18:35:54.793589+0000","last_peered":"2023-06-23T18:35:54.793589+0000","last_clean":"2023-06-23T18:35:54.793589+0000","last_became_active":"2023-06-23T18:35:48.735621+0000","last_became_peered":"2023-06-23T18:35:48.735621+0000","last_unstale":"2023-06-23T18:35:54.793589+0000","last_undegraded":"2023-06-23T18:35:54.793589+0000","last_fullsized":"2023-06-23T18:35:54.793589+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:30:39.104167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527092+0000","last_change":"2023-06-23T18:35:53.131817+0000","last_active":"2023-06-23T18:35:54.527092+0000","last_peered":"2023-06-23T18:35:54.527092+0000","last_clean":"2023-06-23T18:35:54.527092+0000","last_became_active":"2023-06-23T18:35:53.131549+0000","last_became_peered":"2023-06-23T18:35:53.131549+0000","last_unstale":"2023-06-23T18:35:54.527092+0000","last_undegraded":"2023-06-23T18:35:54.527092+0000","last_fullsized":"2023-06-23T18:35:54.527092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:04:00.010842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527627+0000","last_change":"2023-06-23T18:35:53.531485+0000","last_active":"2023-06-23T18:35:54.527627+0000","last_peered":"2023-06-23T18:35:54.527627+0000","last_clean":"2023-06-23T18:35:54.527627+0000","last_became_active":"2023-06-23T18:35:53.531276+0000","last_became_peered":"2023-06-23T18:35:53.531276+0000","last_unstale":"2023-06-23T18:35:54.527627+0000","last_undegraded":"2023-06-23T18:35:54.527627+0000","last_fullsized":"2023-06-23T18:35:54.527627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793552+0000","last_change":"2023-06-23T18:35:48.736449+0000","last_active":"2023-06-23T18:35:54.793552+0000","last_peered":"2023-06-23T18:35:54.793552+0000","last_clean":"2023-06-23T18:35:54.793552+0000","last_became_active":"2023-06-23T18:35:48.735664+0000","last_became_peered":"2023-06-23T18:35:48.735664+0000","last_unstale":"2023-06-23T18:35:54.793552+0000","last_undegraded":"2023-06-23T18:35:54.793552+0000","last_fullsized":"2023-06-23T18:35:54.793552+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:24.194443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288399+0000","last_change":"2023-06-23T18:35:53.127210+0000","last_active":"2023-06-23T18:35:55.288399+0000","last_peered":"2023-06-23T18:35:55.288399+0000","last_clean":"2023-06-23T18:35:55.288399+0000","last_became_active":"2023-06-23T18:35:53.126894+0000","last_became_peered":"2023-06-23T18:35:53.126894+0000","last_unstale":"2023-06-23T18:35:55.288399+0000","last_undegraded":"2023-06-23T18:35:55.288399+0000","last_fullsized":"2023-06-23T18:35:55.288399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:15.867340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257494+0000","last_change":"2023-06-23T18:35:53.539026+0000","last_active":"2023-06-23T18:35:55.257494+0000","last_peered":"2023-06-23T18:35:55.257494+0000","last_clean":"2023-06-23T18:35:55.257494+0000","last_became_active":"2023-06-23T18:35:53.538711+0000","last_became_peered":"2023-06-23T18:35:53.538711+0000","last_unstale":"2023-06-23T18:35:55.257494+0000","last_undegraded":"2023-06-23T18:35:55.257494+0000","last_fullsized":"2023-06-23T18:35:55.257494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:00.677969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526508+0000","last_change":"2023-06-23T18:35:48.735713+0000","last_active":"2023-06-23T18:35:54.526508+0000","last_peered":"2023-06-23T18:35:54.526508+0000","last_clean":"2023-06-23T18:35:54.526508+0000","last_became_active":"2023-06-23T18:35:48.735236+0000","last_became_peered":"2023-06-23T18:35:48.735236+0000","last_unstale":"2023-06-23T18:35:54.526508+0000","last_undegraded":"2023-06-23T18:35:54.526508+0000","last_fullsized":"2023-06-23T18:35:54.526508+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:22.822277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526010+0000","last_change":"2023-06-23T18:35:53.131500+0000","last_active":"2023-06-23T18:35:54.526010+0000","last_peered":"2023-06-23T18:35:54.526010+0000","last_clean":"2023-06-23T18:35:54.526010+0000","last_became_active":"2023-06-23T18:35:53.131222+0000","last_became_peered":"2023-06-23T18:35:53.131222+0000","last_unstale":"2023-06-23T18:35:54.526010+0000","last_undegraded":"2023-06-23T18:35:54.526010+0000","last_fullsized":"2023-06-23T18:35:54.526010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:38.016680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527523+0000","last_change":"2023-06-23T18:35:53.532882+0000","last_active":"2023-06-23T18:35:54.527523+0000","last_peered":"2023-06-23T18:35:54.527523+0000","last_clean":"2023-06-23T18:35:54.527523+0000","last_became_active":"2023-06-23T18:35:53.532615+0000","last_became_peered":"2023-06-23T18:35:53.532615+0000","last_unstale":"2023-06-23T18:35:54.527523+0000","last_undegraded":"2023-06-23T18:35:54.527523+0000","last_fullsized":"2023-06-23T18:35:54.527523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257469+0000","last_change":"2023-06-23T18:35:48.735631+0000","last_active":"2023-06-23T18:35:55.257469+0000","last_peered":"2023-06-23T18:35:55.257469+0000","last_clean":"2023-06-23T18:35:55.257469+0000","last_became_active":"2023-06-23T18:35:48.735255+0000","last_became_peered":"2023-06-23T18:35:48.735255+0000","last_unstale":"2023-06-23T18:35:55.257469+0000","last_undegraded":"2023-06-23T18:35:55.257469+0000","last_fullsized":"2023-06-23T18:35:55.257469+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:32:56.020041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527397+0000","last_change":"2023-06-23T18:35:53.132237+0000","last_active":"2023-06-23T18:35:54.527397+0000","last_peered":"2023-06-23T18:35:54.527397+0000","last_clean":"2023-06-23T18:35:54.527397+0000","last_became_active":"2023-06-23T18:35:53.131968+0000","last_became_peered":"2023-06-23T18:35:53.131968+0000","last_unstale":"2023-06-23T18:35:54.527397+0000","last_undegraded":"2023-06-23T18:35:54.527397+0000","last_fullsized":"2023-06-23T18:35:54.527397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:27.976038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287780+0000","last_change":"2023-06-23T18:35:53.536675+0000","last_active":"2023-06-23T18:35:55.287780+0000","last_peered":"2023-06-23T18:35:55.287780+0000","last_clean":"2023-06-23T18:35:55.287780+0000","last_became_active":"2023-06-23T18:35:53.535843+0000","last_became_peered":"2023-06-23T18:35:53.535843+0000","last_unstale":"2023-06-23T18:35:55.287780+0000","last_undegraded":"2023-06-23T18:35:55.287780+0000","last_fullsized":"2023-06-23T18:35:55.287780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:07.647324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257359+0000","last_change":"2023-06-23T18:35:47.970359+0000","last_active":"2023-06-23T18:35:55.257359+0000","last_peered":"2023-06-23T18:35:55.257359+0000","last_clean":"2023-06-23T18:35:55.257359+0000","last_became_active":"2023-06-23T18:35:47.970118+0000","last_became_peered":"2023-06-23T18:35:47.970118+0000","last_unstale":"2023-06-23T18:35:55.257359+0000","last_undegraded":"2023-06-23T18:35:55.257359+0000","last_fullsized":"2023-06-23T18:35:55.257359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:35.810772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527072+0000","last_change":"2023-06-23T18:35:53.132066+0000","last_active":"2023-06-23T18:35:54.527072+0000","last_peered":"2023-06-23T18:35:54.527072+0000","last_clean":"2023-06-23T18:35:54.527072+0000","last_became_active":"2023-06-23T18:35:53.131788+0000","last_became_peered":"2023-06-23T18:35:53.131788+0000","last_unstale":"2023-06-23T18:35:54.527072+0000","last_undegraded":"2023-06-23T18:35:54.527072+0000","last_fullsized":"2023-06-23T18:35:54.527072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:30.042205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527690+0000","last_change":"2023-06-23T18:35:53.536630+0000","last_active":"2023-06-23T18:35:54.527690+0000","last_peered":"2023-06-23T18:35:54.527690+0000","last_clean":"2023-06-23T18:35:54.527690+0000","last_became_active":"2023-06-23T18:35:53.536298+0000","last_became_peered":"2023-06-23T18:35:53.536298+0000","last_unstale":"2023-06-23T18:35:54.527690+0000","last_undegraded":"2023-06-23T18:35:54.527690+0000","last_fullsized":"2023-06-23T18:35:54.527690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:16.869108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287850+0000","last_change":"2023-06-23T18:35:53.126505+0000","last_active":"2023-06-23T18:35:55.287850+0000","last_peered":"2023-06-23T18:35:55.287850+0000","last_clean":"2023-06-23T18:35:55.287850+0000","last_became_active":"2023-06-23T18:35:53.126226+0000","last_became_peered":"2023-06-23T18:35:53.126226+0000","last_unstale":"2023-06-23T18:35:55.287850+0000","last_undegraded":"2023-06-23T18:35:55.287850+0000","last_fullsized":"2023-06-23T18:35:55.287850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:40.964533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287939+0000","last_change":"2023-06-23T18:35:53.533402+0000","last_active":"2023-06-23T18:35:55.287939+0000","last_peered":"2023-06-23T18:35:55.287939+0000","last_clean":"2023-06-23T18:35:55.287939+0000","last_became_active":"2023-06-23T18:35:53.533163+0000","last_became_peered":"2023-06-23T18:35:53.533163+0000","last_unstale":"2023-06-23T18:35:55.287939+0000","last_undegraded":"2023-06-23T18:35:55.287939+0000","last_fullsized":"2023-06-23T18:35:55.287939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:01:29.821609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526906+0000","last_change":"2023-06-23T18:35:53.123684+0000","last_active":"2023-06-23T18:35:54.526906+0000","last_peered":"2023-06-23T18:35:54.526906+0000","last_clean":"2023-06-23T18:35:54.526906+0000","last_became_active":"2023-06-23T18:35:53.123369+0000","last_became_peered":"2023-06-23T18:35:53.123369+0000","last_unstale":"2023-06-23T18:35:54.526906+0000","last_undegraded":"2023-06-23T18:35:54.526906+0000","last_fullsized":"2023-06-23T18:35:54.526906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:18.604812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525869+0000","last_change":"2023-06-23T18:35:53.534162+0000","last_active":"2023-06-23T18:35:54.525869+0000","last_peered":"2023-06-23T18:35:54.525869+0000","last_clean":"2023-06-23T18:35:54.525869+0000","last_became_active":"2023-06-23T18:35:53.533853+0000","last_became_peered":"2023-06-23T18:35:53.533853+0000","last_unstale":"2023-06-23T18:35:54.525869+0000","last_undegraded":"2023-06-23T18:35:54.525869+0000","last_fullsized":"2023-06-23T18:35:54.525869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:30.583311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527209+0000","last_change":"2023-06-23T18:35:53.123532+0000","last_active":"2023-06-23T18:35:54.527209+0000","last_peered":"2023-06-23T18:35:54.527209+0000","last_clean":"2023-06-23T18:35:54.527209+0000","last_became_active":"2023-06-23T18:35:53.123228+0000","last_became_peered":"2023-06-23T18:35:53.123228+0000","last_unstale":"2023-06-23T18:35:54.527209+0000","last_undegraded":"2023-06-23T18:35:54.527209+0000","last_fullsized":"2023-06-23T18:35:54.527209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:29.581544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527107+0000","last_change":"2023-06-23T18:35:53.533902+0000","last_active":"2023-06-23T18:35:54.527107+0000","last_peered":"2023-06-23T18:35:54.527107+0000","last_clean":"2023-06-23T18:35:54.527107+0000","last_became_active":"2023-06-23T18:35:53.533563+0000","last_became_peered":"2023-06-23T18:35:53.533563+0000","last_unstale":"2023-06-23T18:35:54.527107+0000","last_undegraded":"2023-06-23T18:35:54.527107+0000","last_fullsized":"2023-06-23T18:35:54.527107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:55:12.340901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288866+0000","last_change":"2023-06-23T18:35:53.126195+0000","last_active":"2023-06-23T18:35:55.288866+0000","last_peered":"2023-06-23T18:35:55.288866+0000","last_clean":"2023-06-23T18:35:55.288866+0000","last_became_active":"2023-06-23T18:35:53.125886+0000","last_became_peered":"2023-06-23T18:35:53.125886+0000","last_unstale":"2023-06-23T18:35:55.288866+0000","last_undegraded":"2023-06-23T18:35:55.288866+0000","last_fullsized":"2023-06-23T18:35:55.288866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:57.730157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288616+0000","last_change":"2023-06-23T18:35:53.534747+0000","last_active":"2023-06-23T18:35:55.288616+0000","last_peered":"2023-06-23T18:35:55.288616+0000","last_clean":"2023-06-23T18:35:55.288616+0000","last_became_active":"2023-06-23T18:35:53.534495+0000","last_became_peered":"2023-06-23T18:35:53.534495+0000","last_unstale":"2023-06-23T18:35:55.288616+0000","last_undegraded":"2023-06-23T18:35:55.288616+0000","last_fullsized":"2023-06-23T18:35:55.288616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:30.583311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288416+0000","last_change":"2023-06-23T18:35:53.130647+0000","last_active":"2023-06-23T18:35:55.288416+0000","last_peered":"2023-06-23T18:35:55.288416+0000","last_clean":"2023-06-23T18:35:55.288416+0000","last_became_active":"2023-06-23T18:35:53.129738+0000","last_became_peered":"2023-06-23T18:35:53.129738+0000","last_unstale":"2023-06-23T18:35:55.288416+0000","last_undegraded":"2023-06-23T18:35:55.288416+0000","last_fullsized":"2023-06-23T18:35:55.288416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:16.518353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525676+0000","last_change":"2023-06-23T18:35:53.540648+0000","last_active":"2023-06-23T18:35:54.525676+0000","last_peered":"2023-06-23T18:35:54.525676+0000","last_clean":"2023-06-23T18:35:54.525676+0000","last_became_active":"2023-06-23T18:35:53.540380+0000","last_became_peered":"2023-06-23T18:35:53.540380+0000","last_unstale":"2023-06-23T18:35:54.525676+0000","last_undegraded":"2023-06-23T18:35:54.525676+0000","last_fullsized":"2023-06-23T18:35:54.525676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257206+0000","last_change":"2023-06-23T18:35:53.539029+0000","last_active":"2023-06-23T18:35:55.257206+0000","last_peered":"2023-06-23T18:35:55.257206+0000","last_clean":"2023-06-23T18:35:55.257206+0000","last_became_active":"2023-06-23T18:35:53.538640+0000","last_became_peered":"2023-06-23T18:35:53.538640+0000","last_unstale":"2023-06-23T18:35:55.257206+0000","last_undegraded":"2023-06-23T18:35:55.257206+0000","last_fullsized":"2023-06-23T18:35:55.257206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:52.658179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288459+0000","last_change":"2023-06-23T18:35:53.541022+0000","last_active":"2023-06-23T18:35:55.288459+0000","last_peered":"2023-06-23T18:35:55.288459+0000","last_clean":"2023-06-23T18:35:55.288459+0000","last_became_active":"2023-06-23T18:35:53.540744+0000","last_became_peered":"2023-06-23T18:35:53.540744+0000","last_unstale":"2023-06-23T18:35:55.288459+0000","last_undegraded":"2023-06-23T18:35:55.288459+0000","last_fullsized":"2023-06-23T18:35:55.288459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:31.043972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526923+0000","last_change":"2023-06-23T18:35:53.540406+0000","last_active":"2023-06-23T18:35:54.526923+0000","last_peered":"2023-06-23T18:35:54.526923+0000","last_clean":"2023-06-23T18:35:54.526923+0000","last_became_active":"2023-06-23T18:35:53.540125+0000","last_became_peered":"2023-06-23T18:35:53.540125+0000","last_unstale":"2023-06-23T18:35:54.526923+0000","last_undegraded":"2023-06-23T18:35:54.526923+0000","last_fullsized":"2023-06-23T18:35:54.526923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:45:17.669839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288781+0000","last_change":"2023-06-23T18:35:53.542899+0000","last_active":"2023-06-23T18:35:55.288781+0000","last_peered":"2023-06-23T18:35:55.288781+0000","last_clean":"2023-06-23T18:35:55.288781+0000","last_became_active":"2023-06-23T18:35:53.541798+0000","last_became_peered":"2023-06-23T18:35:53.541798+0000","last_unstale":"2023-06-23T18:35:55.288781+0000","last_undegraded":"2023-06-23T18:35:55.288781+0000","last_fullsized":"2023-06-23T18:35:55.288781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:30.349972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527379+0000","last_change":"2023-06-23T18:35:53.532923+0000","last_active":"2023-06-23T18:35:54.527379+0000","last_peered":"2023-06-23T18:35:54.527379+0000","last_clean":"2023-06-23T18:35:54.527379+0000","last_became_active":"2023-06-23T18:35:53.532674+0000","last_became_peered":"2023-06-23T18:35:53.532674+0000","last_unstale":"2023-06-23T18:35:54.527379+0000","last_undegraded":"2023-06-23T18:35:54.527379+0000","last_fullsized":"2023-06-23T18:35:54.527379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:17.016870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":823296,"data_stored":665215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.798}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9550000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":74428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":798720,"data_stored":664796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":73649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":200704,"data_stored":73067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":208896,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1559999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":76048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":75080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:36:02.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:36:02.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.442+0000 7f0c7f1ee700 1 Processor -- start 2023-06-23T18:36:02.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.442+0000 7f0c7f1ee700 1 -- start start 2023-06-23T18:36:02.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c7f1ee700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0c78159170 0x7f0c780cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c7f1ee700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0c780cd900 0x7f0c780ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c7f1ee700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c780ce620 0x7f0c78071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c7f1ee700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c780cdd80 con 0x7f0c780cd900 2023-06-23T18:36:02.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c7f1ee700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c7815cd50 con 0x7f0c78159170 2023-06-23T18:36:02.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c7f1ee700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7815cfa0 con 0x7f0c780ce620 2023-06-23T18:36:02.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c77fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0c780cd900 0x7f0c780ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c77fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0c780cd900 0x7f0c780ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39586/0 (socket says 172.21.15.79:39586) 2023-06-23T18:36:02.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c77fff700 1 -- 172.21.15.79:0/2824948416 learned_addr learned my addr 172.21.15.79:0/2824948416 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:02.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c7d78b700 1 --2- 172.21.15.79:0/2824948416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c780ce620 0x7f0c78071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c7cf8a700 1 --2- 172.21.15.79:0/2824948416 >> v2:172.21.15.155:3300/0 conn(0x7f0c78159170 0x7f0c780cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c77fff700 1 -- 172.21.15.79:0/2824948416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c780ce620 msgr2=0x7f0c78071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:02.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c77fff700 1 --2- 172.21.15.79:0/2824948416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c780ce620 0x7f0c78071490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c77fff700 1 -- 172.21.15.79:0/2824948416 >> v2:172.21.15.155:3300/0 conn(0x7f0c78159170 msgr2=0x7f0c780cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:02.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c77fff700 1 --2- 172.21.15.79:0/2824948416 >> v2:172.21.15.155:3300/0 conn(0x7f0c78159170 0x7f0c780cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c77fff700 1 -- 172.21.15.79:0/2824948416 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7815d280 con 0x7f0c780cd900 2023-06-23T18:36:02.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c77fff700 1 --2- 172.21.15.79:0/2824948416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c780cd900 0x7f0c780ce0e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0c78070aa0 tx=0x7f0c6400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54182d18ae524e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:02.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c777fe700 1 -- 172.21.15.79:0/2824948416 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400dd30 con 0x7f0c780cd900 2023-06-23T18:36:02.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c777fe700 1 -- 172.21.15.79:0/2824948416 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6400de90 con 0x7f0c780cd900 2023-06-23T18:36:02.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.443+0000 7f0c777fe700 1 -- 172.21.15.79:0/2824948416 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64010690 con 0x7f0c780cd900 2023-06-23T18:36:02.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.444+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/2824948416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c780cd900 msgr2=0x7f0c780ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.444+0000 7f0c7f1ee700 1 --2- 172.21.15.79:0/2824948416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c780cd900 0x7f0c780ce0e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0c78070aa0 tx=0x7f0c6400be70 comp rx=0 tx=0).stop 2023-06-23T18:36:02.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.444+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/2824948416 shutdown_connections 2023-06-23T18:36:02.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.444+0000 7f0c7f1ee700 1 --2- 172.21.15.79:0/2824948416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c780ce620 0x7f0c78071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.444+0000 7f0c7f1ee700 1 --2- 172.21.15.79:0/2824948416 >> v2:172.21.15.155:3300/0 conn(0x7f0c78159170 0x7f0c780cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.444+0000 7f0c7f1ee700 1 --2- 172.21.15.79:0/2824948416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c780cd900 0x7f0c780ce0e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.444+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/2824948416 >> 172.21.15.79:0/2824948416 conn(0x7f0c7806d660 msgr2=0x7f0c7806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:02.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.444+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/2824948416 shutdown_connections 2023-06-23T18:36:02.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.444+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/2824948416 wait complete. 2023-06-23T18:36:02.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.446+0000 7f0c7f1ee700 1 Processor -- start 2023-06-23T18:36:02.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.446+0000 7f0c7f1ee700 1 -- start start 2023-06-23T18:36:02.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c7f1ee700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c780cd900 0x7f0c781531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c7f1ee700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c780ce620 0x7f0c78154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c7f1ee700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c78159170 0x7f0c781577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c7f1ee700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7815b400 con 0x7f0c780ce620 2023-06-23T18:36:02.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c7f1ee700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7815b820 con 0x7f0c780cd900 2023-06-23T18:36:02.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c7f1ee700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7815a780 con 0x7f0c78159170 2023-06-23T18:36:02.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c77fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c780ce620 0x7f0c78154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c77fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c780ce620 0x7f0c78154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39594/0 (socket says 172.21.15.79:39594) 2023-06-23T18:36:02.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c77fff700 1 -- 172.21.15.79:0/1025539952 learned_addr learned my addr 172.21.15.79:0/1025539952 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:02.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c77fff700 1 -- 172.21.15.79:0/1025539952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c78159170 msgr2=0x7f0c781577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c77fff700 1 --2- 172.21.15.79:0/1025539952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c78159170 0x7f0c781577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c77fff700 1 -- 172.21.15.79:0/1025539952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c780cd900 msgr2=0x7f0c781531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c77fff700 1 --2- 172.21.15.79:0/1025539952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c780cd900 0x7f0c781531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c77fff700 1 -- 172.21.15.79:0/1025539952 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6400b7f0 con 0x7f0c780ce620 2023-06-23T18:36:02.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c77fff700 1 --2- 172.21.15.79:0/1025539952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c780ce620 0x7f0c78154160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0c78070bc0 tx=0x7f0c64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:02.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c757fa700 1 -- 172.21.15.79:0/1025539952 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6401f070 con 0x7f0c780ce620 2023-06-23T18:36:02.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c757fa700 1 -- 172.21.15.79:0/1025539952 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c64003b20 con 0x7f0c780ce620 2023-06-23T18:36:02.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c757fa700 1 -- 172.21.15.79:0/1025539952 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64010470 con 0x7f0c780ce620 2023-06-23T18:36:02.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/1025539952 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7815aa60 con 0x7f0c780ce620 2023-06-23T18:36:02.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.447+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/1025539952 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7815b020 con 0x7f0c780ce620 2023-06-23T18:36:02.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.448+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/1025539952 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c780cbde0 con 0x7f0c780ce620 2023-06-23T18:36:02.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.449+0000 7f0c757fa700 1 -- 172.21.15.79:0/1025539952 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0c64019400 con 0x7f0c780ce620 2023-06-23T18:36:02.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.449+0000 7f0c757fa700 1 --2- 172.21.15.79:0/1025539952 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0c5c06ace0 0x7f0c5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.449+0000 7f0c757fa700 1 -- 172.21.15.79:0/1025539952 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0c6408d0e0 con 0x7f0c780ce620 2023-06-23T18:36:02.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.449+0000 7f0c7cf8a700 1 --2- 172.21.15.79:0/1025539952 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0c5c06ace0 0x7f0c5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.450+0000 7f0c7cf8a700 1 --2- 172.21.15.79:0/1025539952 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0c5c06ace0 0x7f0c5c06d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00b850 tx=0x7f0c6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:02.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.454+0000 7f0c757fa700 1 -- 172.21.15.79:0/1025539952 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0c64058180 con 0x7f0c780ce620 2023-06-23T18:36:02.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.635+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/1025539952 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c7815d840 con 0x7f0c5c06ace0 2023-06-23T18:36:02.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.643+0000 7f0c757fa700 1 -- 172.21.15.79:0/1025539952 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286836 (secure 0 0 0) 0x7f0c6c00b4f0 con 0x7f0c5c06ace0 2023-06-23T18:36:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:02.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.647+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/1025539952 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0c5c06ace0 msgr2=0x7f0c5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.647+0000 7f0c7f1ee700 1 --2- 172.21.15.79:0/1025539952 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0c5c06ace0 0x7f0c5c06d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00b850 tx=0x7f0c6c008040 comp rx=0 tx=0).stop 2023-06-23T18:36:02.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.647+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/1025539952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c780ce620 msgr2=0x7f0c78154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.647+0000 7f0c7f1ee700 1 --2- 172.21.15.79:0/1025539952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c780ce620 0x7f0c78154160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0c78070bc0 tx=0x7f0c64004750 comp rx=0 tx=0).stop 2023-06-23T18:36:02.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.647+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/1025539952 shutdown_connections 2023-06-23T18:36:02.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.647+0000 7f0c7f1ee700 1 --2- 172.21.15.79:0/1025539952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c78159170 0x7f0c781577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.647+0000 7f0c7f1ee700 1 --2- 172.21.15.79:0/1025539952 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0c5c06ace0 0x7f0c5c06d190 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.647+0000 7f0c7f1ee700 1 --2- 172.21.15.79:0/1025539952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c780cd900 0x7f0c781531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.647+0000 7f0c7f1ee700 1 --2- 172.21.15.79:0/1025539952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c780ce620 0x7f0c78154160 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.647+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/1025539952 >> 172.21.15.79:0/1025539952 conn(0x7f0c7806d660 msgr2=0x7f0c780cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:02.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.647+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/1025539952 shutdown_connections 2023-06-23T18:36:02.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.647+0000 7f0c7f1ee700 1 -- 172.21.15.79:0/1025539952 wait complete. 2023-06-23T18:36:02.655 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-23T18:36:02.663 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-06-23T18:36:00.696780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218428,"kb_used_data":2812,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756292,"statfs":{"total":773094113280,"available":772870443008,"internally_reserved":0,"allocated":2879488,"data_stored":1776801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.585774"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287219+0000","last_change":"2023-06-23T18:35:53.534999+0000","last_active":"2023-06-23T18:35:55.287219+0000","last_peered":"2023-06-23T18:35:55.287219+0000","last_clean":"2023-06-23T18:35:55.287219+0000","last_became_active":"2023-06-23T18:35:53.534775+0000","last_became_peered":"2023-06-23T18:35:53.534775+0000","last_unstale":"2023-06-23T18:35:55.287219+0000","last_undegraded":"2023-06-23T18:35:55.287219+0000","last_fullsized":"2023-06-23T18:35:55.287219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:15:42.949792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527064+0000","last_change":"2023-06-23T18:35:53.537237+0000","last_active":"2023-06-23T18:35:54.527064+0000","last_peered":"2023-06-23T18:35:54.527064+0000","last_clean":"2023-06-23T18:35:54.527064+0000","last_became_active":"2023-06-23T18:35:53.536900+0000","last_became_peered":"2023-06-23T18:35:53.536900+0000","last_unstale":"2023-06-23T18:35:54.527064+0000","last_undegraded":"2023-06-23T18:35:54.527064+0000","last_fullsized":"2023-06-23T18:35:54.527064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:17.520121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526243+0000","last_change":"2023-06-23T18:35:53.527037+0000","last_active":"2023-06-23T18:35:54.526243+0000","last_peered":"2023-06-23T18:35:54.526243+0000","last_clean":"2023-06-23T18:35:54.526243+0000","last_became_active":"2023-06-23T18:35:53.526655+0000","last_became_peered":"2023-06-23T18:35:53.526655+0000","last_unstale":"2023-06-23T18:35:54.526243+0000","last_undegraded":"2023-06-23T18:35:54.526243+0000","last_fullsized":"2023-06-23T18:35:54.526243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:30.349972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257396+0000","last_change":"2023-06-23T18:35:53.540867+0000","last_active":"2023-06-23T18:35:55.257396+0000","last_peered":"2023-06-23T18:35:55.257396+0000","last_clean":"2023-06-23T18:35:55.257396+0000","last_became_active":"2023-06-23T18:35:53.540572+0000","last_became_peered":"2023-06-23T18:35:53.540572+0000","last_unstale":"2023-06-23T18:35:55.257396+0000","last_undegraded":"2023-06-23T18:35:55.257396+0000","last_fullsized":"2023-06-23T18:35:55.257396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:28.977806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287682+0000","last_change":"2023-06-23T18:35:53.535027+0000","last_active":"2023-06-23T18:35:55.287682+0000","last_peered":"2023-06-23T18:35:55.287682+0000","last_clean":"2023-06-23T18:35:55.287682+0000","last_became_active":"2023-06-23T18:35:53.534793+0000","last_became_peered":"2023-06-23T18:35:53.534793+0000","last_unstale":"2023-06-23T18:35:55.287682+0000","last_undegraded":"2023-06-23T18:35:55.287682+0000","last_fullsized":"2023-06-23T18:35:55.287682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:17.016870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526443+0000","last_change":"2023-06-23T18:35:53.524551+0000","last_active":"2023-06-23T18:35:54.526443+0000","last_peered":"2023-06-23T18:35:54.526443+0000","last_clean":"2023-06-23T18:35:54.526443+0000","last_became_active":"2023-06-23T18:35:53.524271+0000","last_became_peered":"2023-06-23T18:35:53.524271+0000","last_unstale":"2023-06-23T18:35:54.526443+0000","last_undegraded":"2023-06-23T18:35:54.526443+0000","last_fullsized":"2023-06-23T18:35:54.526443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:08:04.256786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793931+0000","last_change":"2023-06-23T18:35:53.540542+0000","last_active":"2023-06-23T18:35:54.793931+0000","last_peered":"2023-06-23T18:35:54.793931+0000","last_clean":"2023-06-23T18:35:54.793931+0000","last_became_active":"2023-06-23T18:35:53.539978+0000","last_became_peered":"2023-06-23T18:35:53.539978+0000","last_unstale":"2023-06-23T18:35:54.793931+0000","last_undegraded":"2023-06-23T18:35:54.793931+0000","last_fullsized":"2023-06-23T18:35:54.793931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:57:46.218466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525592+0000","last_change":"2023-06-23T18:35:53.533541+0000","last_active":"2023-06-23T18:35:54.525592+0000","last_peered":"2023-06-23T18:35:54.525592+0000","last_clean":"2023-06-23T18:35:54.525592+0000","last_became_active":"2023-06-23T18:35:53.531085+0000","last_became_peered":"2023-06-23T18:35:53.531085+0000","last_unstale":"2023-06-23T18:35:54.525592+0000","last_undegraded":"2023-06-23T18:35:54.525592+0000","last_fullsized":"2023-06-23T18:35:54.525592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:52.658179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288165+0000","last_change":"2023-06-23T18:35:53.534716+0000","last_active":"2023-06-23T18:35:55.288165+0000","last_peered":"2023-06-23T18:35:55.288165+0000","last_clean":"2023-06-23T18:35:55.288165+0000","last_became_active":"2023-06-23T18:35:53.534450+0000","last_became_peered":"2023-06-23T18:35:53.534450+0000","last_unstale":"2023-06-23T18:35:55.288165+0000","last_undegraded":"2023-06-23T18:35:55.288165+0000","last_fullsized":"2023-06-23T18:35:55.288165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527020+0000","last_change":"2023-06-23T18:35:53.533580+0000","last_active":"2023-06-23T18:35:54.527020+0000","last_peered":"2023-06-23T18:35:54.527020+0000","last_clean":"2023-06-23T18:35:54.527020+0000","last_became_active":"2023-06-23T18:35:53.533273+0000","last_became_peered":"2023-06-23T18:35:53.533273+0000","last_unstale":"2023-06-23T18:35:54.527020+0000","last_undegraded":"2023-06-23T18:35:54.527020+0000","last_fullsized":"2023-06-23T18:35:54.527020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526443+0000","last_change":"2023-06-23T18:35:53.537638+0000","last_active":"2023-06-23T18:35:54.526443+0000","last_peered":"2023-06-23T18:35:54.526443+0000","last_clean":"2023-06-23T18:35:54.526443+0000","last_became_active":"2023-06-23T18:35:53.537351+0000","last_became_peered":"2023-06-23T18:35:53.537351+0000","last_unstale":"2023-06-23T18:35:54.526443+0000","last_undegraded":"2023-06-23T18:35:54.526443+0000","last_fullsized":"2023-06-23T18:35:54.526443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:25:31.659636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288045+0000","last_change":"2023-06-23T18:35:53.530072+0000","last_active":"2023-06-23T18:35:55.288045+0000","last_peered":"2023-06-23T18:35:55.288045+0000","last_clean":"2023-06-23T18:35:55.288045+0000","last_became_active":"2023-06-23T18:35:53.529676+0000","last_became_peered":"2023-06-23T18:35:53.529676+0000","last_unstale":"2023-06-23T18:35:55.288045+0000","last_undegraded":"2023-06-23T18:35:55.288045+0000","last_fullsized":"2023-06-23T18:35:55.288045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:04:21.490439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287613+0000","last_change":"2023-06-23T18:35:53.535447+0000","last_active":"2023-06-23T18:35:55.287613+0000","last_peered":"2023-06-23T18:35:55.287613+0000","last_clean":"2023-06-23T18:35:55.287613+0000","last_became_active":"2023-06-23T18:35:53.535211+0000","last_became_peered":"2023-06-23T18:35:53.535211+0000","last_unstale":"2023-06-23T18:35:55.287613+0000","last_undegraded":"2023-06-23T18:35:55.287613+0000","last_fullsized":"2023-06-23T18:35:55.287613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:29:59.349736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526832+0000","last_change":"2023-06-23T18:35:53.530753+0000","last_active":"2023-06-23T18:35:54.526832+0000","last_peered":"2023-06-23T18:35:54.526832+0000","last_clean":"2023-06-23T18:35:54.526832+0000","last_became_active":"2023-06-23T18:35:53.530428+0000","last_became_peered":"2023-06-23T18:35:53.530428+0000","last_unstale":"2023-06-23T18:35:54.526832+0000","last_undegraded":"2023-06-23T18:35:54.526832+0000","last_fullsized":"2023-06-23T18:35:54.526832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:00.677969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794039+0000","last_change":"2023-06-23T18:35:53.540712+0000","last_active":"2023-06-23T18:35:54.794039+0000","last_peered":"2023-06-23T18:35:54.794039+0000","last_clean":"2023-06-23T18:35:54.794039+0000","last_became_active":"2023-06-23T18:35:53.540284+0000","last_became_peered":"2023-06-23T18:35:53.540284+0000","last_unstale":"2023-06-23T18:35:54.794039+0000","last_undegraded":"2023-06-23T18:35:54.794039+0000","last_fullsized":"2023-06-23T18:35:54.794039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:12.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525689+0000","last_change":"2023-06-23T18:35:53.524624+0000","last_active":"2023-06-23T18:35:54.525689+0000","last_peered":"2023-06-23T18:35:54.525689+0000","last_clean":"2023-06-23T18:35:54.525689+0000","last_became_active":"2023-06-23T18:35:53.524336+0000","last_became_peered":"2023-06-23T18:35:53.524336+0000","last_unstale":"2023-06-23T18:35:54.525689+0000","last_undegraded":"2023-06-23T18:35:54.525689+0000","last_fullsized":"2023-06-23T18:35:54.525689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:16.869108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288289+0000","last_change":"2023-06-23T18:35:53.534752+0000","last_active":"2023-06-23T18:35:55.288289+0000","last_peered":"2023-06-23T18:35:55.288289+0000","last_clean":"2023-06-23T18:35:55.288289+0000","last_became_active":"2023-06-23T18:35:53.534491+0000","last_became_peered":"2023-06-23T18:35:53.534491+0000","last_unstale":"2023-06-23T18:35:55.288289+0000","last_undegraded":"2023-06-23T18:35:55.288289+0000","last_fullsized":"2023-06-23T18:35:55.288289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:16.869108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526631+0000","last_change":"2023-06-23T18:35:53.530226+0000","last_active":"2023-06-23T18:35:54.526631+0000","last_peered":"2023-06-23T18:35:54.526631+0000","last_clean":"2023-06-23T18:35:54.526631+0000","last_became_active":"2023-06-23T18:35:53.529956+0000","last_became_peered":"2023-06-23T18:35:53.529956+0000","last_unstale":"2023-06-23T18:35:54.526631+0000","last_undegraded":"2023-06-23T18:35:54.526631+0000","last_fullsized":"2023-06-23T18:35:54.526631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:12.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287300+0000","last_change":"2023-06-23T18:35:53.540646+0000","last_active":"2023-06-23T18:35:55.287300+0000","last_peered":"2023-06-23T18:35:55.287300+0000","last_clean":"2023-06-23T18:35:55.287300+0000","last_became_active":"2023-06-23T18:35:53.540370+0000","last_became_peered":"2023-06-23T18:35:53.540370+0000","last_unstale":"2023-06-23T18:35:55.287300+0000","last_undegraded":"2023-06-23T18:35:55.287300+0000","last_fullsized":"2023-06-23T18:35:55.287300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:38:52.875168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526208+0000","last_change":"2023-06-23T18:35:53.539590+0000","last_active":"2023-06-23T18:35:54.526208+0000","last_peered":"2023-06-23T18:35:54.526208+0000","last_clean":"2023-06-23T18:35:54.526208+0000","last_became_active":"2023-06-23T18:35:53.539298+0000","last_became_peered":"2023-06-23T18:35:53.539298+0000","last_unstale":"2023-06-23T18:35:54.526208+0000","last_undegraded":"2023-06-23T18:35:54.526208+0000","last_fullsized":"2023-06-23T18:35:54.526208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:54:07.361405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794125+0000","last_change":"2023-06-23T18:35:53.543302+0000","last_active":"2023-06-23T18:35:54.794125+0000","last_peered":"2023-06-23T18:35:54.794125+0000","last_clean":"2023-06-23T18:35:54.794125+0000","last_became_active":"2023-06-23T18:35:53.542901+0000","last_became_peered":"2023-06-23T18:35:53.542901+0000","last_unstale":"2023-06-23T18:35:54.794125+0000","last_undegraded":"2023-06-23T18:35:54.794125+0000","last_fullsized":"2023-06-23T18:35:54.794125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:26:44.849536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287754+0000","last_change":"2023-06-23T18:35:53.525002+0000","last_active":"2023-06-23T18:35:55.287754+0000","last_peered":"2023-06-23T18:35:55.287754+0000","last_clean":"2023-06-23T18:35:55.287754+0000","last_became_active":"2023-06-23T18:35:53.524694+0000","last_became_peered":"2023-06-23T18:35:53.524694+0000","last_unstale":"2023-06-23T18:35:55.287754+0000","last_undegraded":"2023-06-23T18:35:55.287754+0000","last_fullsized":"2023-06-23T18:35:55.287754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:28.977806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288309+0000","last_change":"2023-06-23T18:35:53.532165+0000","last_active":"2023-06-23T18:35:55.288309+0000","last_peered":"2023-06-23T18:35:55.288309+0000","last_clean":"2023-06-23T18:35:55.288309+0000","last_became_active":"2023-06-23T18:35:53.531883+0000","last_became_peered":"2023-06-23T18:35:53.531883+0000","last_unstale":"2023-06-23T18:35:55.288309+0000","last_undegraded":"2023-06-23T18:35:55.288309+0000","last_fullsized":"2023-06-23T18:35:55.288309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793736+0000","last_change":"2023-06-23T18:35:53.543926+0000","last_active":"2023-06-23T18:35:54.793736+0000","last_peered":"2023-06-23T18:35:54.793736+0000","last_clean":"2023-06-23T18:35:54.793736+0000","last_became_active":"2023-06-23T18:35:53.543692+0000","last_became_peered":"2023-06-23T18:35:53.543692+0000","last_unstale":"2023-06-23T18:35:54.793736+0000","last_undegraded":"2023-06-23T18:35:54.793736+0000","last_fullsized":"2023-06-23T18:35:54.793736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:19:44.074507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527502+0000","last_change":"2023-06-23T18:35:53.539274+0000","last_active":"2023-06-23T18:35:54.527502+0000","last_peered":"2023-06-23T18:35:54.527502+0000","last_clean":"2023-06-23T18:35:54.527502+0000","last_became_active":"2023-06-23T18:35:53.539000+0000","last_became_peered":"2023-06-23T18:35:53.539000+0000","last_unstale":"2023-06-23T18:35:54.527502+0000","last_undegraded":"2023-06-23T18:35:54.527502+0000","last_fullsized":"2023-06-23T18:35:54.527502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:00.677969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527478+0000","last_change":"2023-06-23T18:35:53.532343+0000","last_active":"2023-06-23T18:35:54.527478+0000","last_peered":"2023-06-23T18:35:54.527478+0000","last_clean":"2023-06-23T18:35:54.527478+0000","last_became_active":"2023-06-23T18:35:53.532137+0000","last_became_peered":"2023-06-23T18:35:53.532137+0000","last_unstale":"2023-06-23T18:35:54.527478+0000","last_undegraded":"2023-06-23T18:35:54.527478+0000","last_fullsized":"2023-06-23T18:35:54.527478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:41.966301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526775+0000","last_change":"2023-06-23T18:35:53.541546+0000","last_active":"2023-06-23T18:35:54.526775+0000","last_peered":"2023-06-23T18:35:54.526775+0000","last_clean":"2023-06-23T18:35:54.526775+0000","last_became_active":"2023-06-23T18:35:53.541285+0000","last_became_peered":"2023-06-23T18:35:53.541285+0000","last_unstale":"2023-06-23T18:35:54.526775+0000","last_undegraded":"2023-06-23T18:35:54.526775+0000","last_fullsized":"2023-06-23T18:35:54.526775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:41.966301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527025+0000","last_change":"2023-06-23T18:35:53.130775+0000","last_active":"2023-06-23T18:35:54.527025+0000","last_peered":"2023-06-23T18:35:54.527025+0000","last_clean":"2023-06-23T18:35:54.527025+0000","last_became_active":"2023-06-23T18:35:53.130473+0000","last_became_peered":"2023-06-23T18:35:53.130473+0000","last_unstale":"2023-06-23T18:35:54.527025+0000","last_undegraded":"2023-06-23T18:35:54.527025+0000","last_fullsized":"2023-06-23T18:35:54.527025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:26:43.847769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288817+0000","last_change":"2023-06-23T18:35:53.530941+0000","last_active":"2023-06-23T18:35:55.288817+0000","last_peered":"2023-06-23T18:35:55.288817+0000","last_clean":"2023-06-23T18:35:55.288817+0000","last_became_active":"2023-06-23T18:35:53.530683+0000","last_became_peered":"2023-06-23T18:35:53.530683+0000","last_unstale":"2023-06-23T18:35:55.288817+0000","last_undegraded":"2023-06-23T18:35:55.288817+0000","last_fullsized":"2023-06-23T18:35:55.288817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:19:44.074507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527326+0000","last_change":"2023-06-23T18:35:53.132155+0000","last_active":"2023-06-23T18:35:54.527326+0000","last_peered":"2023-06-23T18:35:54.527326+0000","last_clean":"2023-06-23T18:35:54.527326+0000","last_became_active":"2023-06-23T18:35:53.131785+0000","last_became_peered":"2023-06-23T18:35:53.131785+0000","last_unstale":"2023-06-23T18:35:54.527326+0000","last_undegraded":"2023-06-23T18:35:54.527326+0000","last_fullsized":"2023-06-23T18:35:54.527326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:51.656412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288482+0000","last_change":"2023-06-23T18:35:53.537294+0000","last_active":"2023-06-23T18:35:55.288482+0000","last_peered":"2023-06-23T18:35:55.288482+0000","last_clean":"2023-06-23T18:35:55.288482+0000","last_became_active":"2023-06-23T18:35:53.536977+0000","last_became_peered":"2023-06-23T18:35:53.536977+0000","last_unstale":"2023-06-23T18:35:55.288482+0000","last_undegraded":"2023-06-23T18:35:55.288482+0000","last_fullsized":"2023-06-23T18:35:55.288482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:30.583311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288741+0000","last_change":"2023-06-23T18:35:53.135828+0000","last_active":"2023-06-23T18:35:55.288741+0000","last_peered":"2023-06-23T18:35:55.288741+0000","last_clean":"2023-06-23T18:35:55.288741+0000","last_became_active":"2023-06-23T18:35:53.135453+0000","last_became_peered":"2023-06-23T18:35:53.135453+0000","last_unstale":"2023-06-23T18:35:55.288741+0000","last_undegraded":"2023-06-23T18:35:55.288741+0000","last_fullsized":"2023-06-23T18:35:55.288741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:54:06.359638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257486+0000","last_change":"2023-06-23T18:35:53.535934+0000","last_active":"2023-06-23T18:35:55.257486+0000","last_peered":"2023-06-23T18:35:55.257486+0000","last_clean":"2023-06-23T18:35:55.257486+0000","last_became_active":"2023-06-23T18:35:53.535505+0000","last_became_peered":"2023-06-23T18:35:53.535505+0000","last_unstale":"2023-06-23T18:35:55.257486+0000","last_undegraded":"2023-06-23T18:35:55.257486+0000","last_fullsized":"2023-06-23T18:35:55.257486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288360+0000","last_change":"2023-06-23T18:35:53.130045+0000","last_active":"2023-06-23T18:35:55.288360+0000","last_peered":"2023-06-23T18:35:55.288360+0000","last_clean":"2023-06-23T18:35:55.288360+0000","last_became_active":"2023-06-23T18:35:53.129374+0000","last_became_peered":"2023-06-23T18:35:53.129374+0000","last_unstale":"2023-06-23T18:35:55.288360+0000","last_undegraded":"2023-06-23T18:35:55.288360+0000","last_fullsized":"2023-06-23T18:35:55.288360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:30.042205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288533+0000","last_change":"2023-06-23T18:35:53.540785+0000","last_active":"2023-06-23T18:35:55.288533+0000","last_peered":"2023-06-23T18:35:55.288533+0000","last_clean":"2023-06-23T18:35:55.288533+0000","last_became_active":"2023-06-23T18:35:53.540511+0000","last_became_peered":"2023-06-23T18:35:53.540511+0000","last_unstale":"2023-06-23T18:35:55.288533+0000","last_undegraded":"2023-06-23T18:35:55.288533+0000","last_fullsized":"2023-06-23T18:35:55.288533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:17.520121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526768+0000","last_change":"2023-06-23T18:35:53.133079+0000","last_active":"2023-06-23T18:35:54.526768+0000","last_peered":"2023-06-23T18:35:54.526768+0000","last_clean":"2023-06-23T18:35:54.526768+0000","last_became_active":"2023-06-23T18:35:53.132662+0000","last_became_peered":"2023-06-23T18:35:53.132662+0000","last_unstale":"2023-06-23T18:35:54.526768+0000","last_undegraded":"2023-06-23T18:35:54.526768+0000","last_fullsized":"2023-06-23T18:35:54.526768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:06.645556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288606+0000","last_change":"2023-06-23T18:35:53.541521+0000","last_active":"2023-06-23T18:35:55.288606+0000","last_peered":"2023-06-23T18:35:55.288606+0000","last_clean":"2023-06-23T18:35:55.288606+0000","last_became_active":"2023-06-23T18:35:53.541265+0000","last_became_peered":"2023-06-23T18:35:53.541265+0000","last_unstale":"2023-06-23T18:35:55.288606+0000","last_undegraded":"2023-06-23T18:35:55.288606+0000","last_fullsized":"2023-06-23T18:35:55.288606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:39.018447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793951+0000","last_change":"2023-06-23T18:35:53.131904+0000","last_active":"2023-06-23T18:35:54.793951+0000","last_peered":"2023-06-23T18:35:54.793951+0000","last_clean":"2023-06-23T18:35:54.793951+0000","last_became_active":"2023-06-23T18:35:53.131581+0000","last_became_peered":"2023-06-23T18:35:53.131581+0000","last_unstale":"2023-06-23T18:35:54.793951+0000","last_undegraded":"2023-06-23T18:35:54.793951+0000","last_fullsized":"2023-06-23T18:35:54.793951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:25:30.657869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527297+0000","last_change":"2023-06-23T18:35:53.542076+0000","last_active":"2023-06-23T18:35:54.527297+0000","last_peered":"2023-06-23T18:35:54.527297+0000","last_clean":"2023-06-23T18:35:54.527297+0000","last_became_active":"2023-06-23T18:35:53.541814+0000","last_became_peered":"2023-06-23T18:35:53.541814+0000","last_unstale":"2023-06-23T18:35:54.527297+0000","last_undegraded":"2023-06-23T18:35:54.527297+0000","last_fullsized":"2023-06-23T18:35:54.527297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:40:01.768196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526932+0000","last_change":"2023-06-23T18:35:53.131165+0000","last_active":"2023-06-23T18:35:54.526932+0000","last_peered":"2023-06-23T18:35:54.526932+0000","last_clean":"2023-06-23T18:35:54.526932+0000","last_became_active":"2023-06-23T18:35:53.130890+0000","last_became_peered":"2023-06-23T18:35:53.130890+0000","last_unstale":"2023-06-23T18:35:54.526932+0000","last_undegraded":"2023-06-23T18:35:54.526932+0000","last_fullsized":"2023-06-23T18:35:54.526932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:16:18.754209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526252+0000","last_change":"2023-06-23T18:35:53.541719+0000","last_active":"2023-06-23T18:35:54.526252+0000","last_peered":"2023-06-23T18:35:54.526252+0000","last_clean":"2023-06-23T18:35:54.526252+0000","last_became_active":"2023-06-23T18:35:53.541456+0000","last_became_peered":"2023-06-23T18:35:53.541456+0000","last_unstale":"2023-06-23T18:35:54.526252+0000","last_undegraded":"2023-06-23T18:35:54.526252+0000","last_fullsized":"2023-06-23T18:35:54.526252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:30:45.259696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794066+0000","last_change":"2023-06-23T18:35:53.136124+0000","last_active":"2023-06-23T18:35:54.794066+0000","last_peered":"2023-06-23T18:35:54.794066+0000","last_clean":"2023-06-23T18:35:54.794066+0000","last_became_active":"2023-06-23T18:35:53.135736+0000","last_became_peered":"2023-06-23T18:35:53.135736+0000","last_unstale":"2023-06-23T18:35:54.794066+0000","last_undegraded":"2023-06-23T18:35:54.794066+0000","last_fullsized":"2023-06-23T18:35:54.794066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:45:16.668072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.289031+0000","last_change":"2023-06-23T18:35:53.540586+0000","last_active":"2023-06-23T18:35:55.289031+0000","last_peered":"2023-06-23T18:35:55.289031+0000","last_clean":"2023-06-23T18:35:55.289031+0000","last_became_active":"2023-06-23T18:35:53.540355+0000","last_became_peered":"2023-06-23T18:35:53.540355+0000","last_unstale":"2023-06-23T18:35:55.289031+0000","last_undegraded":"2023-06-23T18:35:55.289031+0000","last_fullsized":"2023-06-23T18:35:55.289031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:02:35.233797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287933+0000","last_change":"2023-06-23T18:35:53.133113+0000","last_active":"2023-06-23T18:35:55.287933+0000","last_peered":"2023-06-23T18:35:55.287933+0000","last_clean":"2023-06-23T18:35:55.287933+0000","last_became_active":"2023-06-23T18:35:53.132833+0000","last_became_peered":"2023-06-23T18:35:53.132833+0000","last_unstale":"2023-06-23T18:35:55.287933+0000","last_undegraded":"2023-06-23T18:35:55.287933+0000","last_fullsized":"2023-06-23T18:35:55.287933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:46:59.676202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527016+0000","last_change":"2023-06-23T18:35:53.538309+0000","last_active":"2023-06-23T18:35:54.527016+0000","last_peered":"2023-06-23T18:35:54.527016+0000","last_clean":"2023-06-23T18:35:54.527016+0000","last_became_active":"2023-06-23T18:35:53.538037+0000","last_became_peered":"2023-06-23T18:35:53.538037+0000","last_unstale":"2023-06-23T18:35:54.527016+0000","last_undegraded":"2023-06-23T18:35:54.527016+0000","last_fullsized":"2023-06-23T18:35:54.527016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793844+0000","last_change":"2023-06-23T18:35:53.131958+0000","last_active":"2023-06-23T18:35:54.793844+0000","last_peered":"2023-06-23T18:35:54.793844+0000","last_clean":"2023-06-23T18:35:54.793844+0000","last_became_active":"2023-06-23T18:35:53.131643+0000","last_became_peered":"2023-06-23T18:35:53.131643+0000","last_unstale":"2023-06-23T18:35:54.793844+0000","last_undegraded":"2023-06-23T18:35:54.793844+0000","last_fullsized":"2023-06-23T18:35:54.793844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:57.730157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288109+0000","last_change":"2023-06-23T18:35:53.533318+0000","last_active":"2023-06-23T18:35:55.288109+0000","last_peered":"2023-06-23T18:35:55.288109+0000","last_clean":"2023-06-23T18:35:55.288109+0000","last_became_active":"2023-06-23T18:35:53.533081+0000","last_became_peered":"2023-06-23T18:35:53.533081+0000","last_unstale":"2023-06-23T18:35:55.288109+0000","last_undegraded":"2023-06-23T18:35:55.288109+0000","last_fullsized":"2023-06-23T18:35:55.288109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257090+0000","last_change":"2023-06-23T18:35:53.135982+0000","last_active":"2023-06-23T18:35:55.257090+0000","last_peered":"2023-06-23T18:35:55.257090+0000","last_clean":"2023-06-23T18:35:55.257090+0000","last_became_active":"2023-06-23T18:35:53.135591+0000","last_became_peered":"2023-06-23T18:35:53.135591+0000","last_unstale":"2023-06-23T18:35:55.257090+0000","last_undegraded":"2023-06-23T18:35:55.257090+0000","last_fullsized":"2023-06-23T18:35:55.257090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:40:00.766429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288988+0000","last_change":"2023-06-23T18:35:53.541883+0000","last_active":"2023-06-23T18:35:55.288988+0000","last_peered":"2023-06-23T18:35:55.288988+0000","last_clean":"2023-06-23T18:35:55.288988+0000","last_became_active":"2023-06-23T18:35:53.541630+0000","last_became_peered":"2023-06-23T18:35:53.541630+0000","last_unstale":"2023-06-23T18:35:55.288988+0000","last_undegraded":"2023-06-23T18:35:55.288988+0000","last_fullsized":"2023-06-23T18:35:55.288988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:08:04.256786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793955+0000","last_change":"2023-06-23T18:35:53.132583+0000","last_active":"2023-06-23T18:35:54.793955+0000","last_peered":"2023-06-23T18:35:54.793955+0000","last_clean":"2023-06-23T18:35:54.793955+0000","last_became_active":"2023-06-23T18:35:53.132142+0000","last_became_peered":"2023-06-23T18:35:53.132142+0000","last_unstale":"2023-06-23T18:35:54.793955+0000","last_undegraded":"2023-06-23T18:35:54.793955+0000","last_fullsized":"2023-06-23T18:35:54.793955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:02:34.232029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526009+0000","last_change":"2023-06-23T18:35:53.537549+0000","last_active":"2023-06-23T18:35:54.526009+0000","last_peered":"2023-06-23T18:35:54.526009+0000","last_clean":"2023-06-23T18:35:54.526009+0000","last_became_active":"2023-06-23T18:35:53.537281+0000","last_became_peered":"2023-06-23T18:35:53.537281+0000","last_unstale":"2023-06-23T18:35:54.526009+0000","last_undegraded":"2023-06-23T18:35:54.526009+0000","last_fullsized":"2023-06-23T18:35:54.526009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:17.016870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.794202+0000","last_change":"2023-06-23T18:35:53.131954+0000","last_active":"2023-06-23T18:35:54.794202+0000","last_peered":"2023-06-23T18:35:54.794202+0000","last_clean":"2023-06-23T18:35:54.794202+0000","last_became_active":"2023-06-23T18:35:53.131624+0000","last_became_peered":"2023-06-23T18:35:53.131624+0000","last_unstale":"2023-06-23T18:35:54.794202+0000","last_undegraded":"2023-06-23T18:35:54.794202+0000","last_fullsized":"2023-06-23T18:35:54.794202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:16:18.754209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525847+0000","last_change":"2023-06-23T18:35:53.535467+0000","last_active":"2023-06-23T18:35:54.525847+0000","last_peered":"2023-06-23T18:35:54.525847+0000","last_clean":"2023-06-23T18:35:54.525847+0000","last_became_active":"2023-06-23T18:35:53.535187+0000","last_became_peered":"2023-06-23T18:35:53.535187+0000","last_unstale":"2023-06-23T18:35:54.525847+0000","last_undegraded":"2023-06-23T18:35:54.525847+0000","last_fullsized":"2023-06-23T18:35:54.525847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:31.043972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793955+0000","last_change":"2023-06-23T18:35:53.132445+0000","last_active":"2023-06-23T18:35:54.793955+0000","last_peered":"2023-06-23T18:35:54.793955+0000","last_clean":"2023-06-23T18:35:54.793955+0000","last_became_active":"2023-06-23T18:35:53.132108+0000","last_became_peered":"2023-06-23T18:35:53.132108+0000","last_unstale":"2023-06-23T18:35:54.793955+0000","last_undegraded":"2023-06-23T18:35:54.793955+0000","last_fullsized":"2023-06-23T18:35:54.793955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:44:02.521500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257388+0000","last_change":"2023-06-23T18:35:53.539086+0000","last_active":"2023-06-23T18:35:55.257388+0000","last_peered":"2023-06-23T18:35:55.257388+0000","last_clean":"2023-06-23T18:35:55.257388+0000","last_became_active":"2023-06-23T18:35:53.538755+0000","last_became_peered":"2023-06-23T18:35:53.538755+0000","last_unstale":"2023-06-23T18:35:55.257388+0000","last_undegraded":"2023-06-23T18:35:55.257388+0000","last_fullsized":"2023-06-23T18:35:55.257388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:07.647324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527056+0000","last_change":"2023-06-23T18:35:53.128100+0000","last_active":"2023-06-23T18:35:54.527056+0000","last_peered":"2023-06-23T18:35:54.527056+0000","last_clean":"2023-06-23T18:35:54.527056+0000","last_became_active":"2023-06-23T18:35:53.127782+0000","last_became_peered":"2023-06-23T18:35:53.127782+0000","last_unstale":"2023-06-23T18:35:54.527056+0000","last_undegraded":"2023-06-23T18:35:54.527056+0000","last_fullsized":"2023-06-23T18:35:54.527056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:15.867340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287463+0000","last_change":"2023-06-23T18:35:53.524781+0000","last_active":"2023-06-23T18:35:55.287463+0000","last_peered":"2023-06-23T18:35:55.287463+0000","last_clean":"2023-06-23T18:35:55.287463+0000","last_became_active":"2023-06-23T18:35:53.524526+0000","last_became_peered":"2023-06-23T18:35:53.524526+0000","last_unstale":"2023-06-23T18:35:55.287463+0000","last_undegraded":"2023-06-23T18:35:55.287463+0000","last_fullsized":"2023-06-23T18:35:55.287463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:40:01.768196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288801+0000","last_change":"2023-06-23T18:35:53.133550+0000","last_active":"2023-06-23T18:35:55.288801+0000","last_peered":"2023-06-23T18:35:55.288801+0000","last_clean":"2023-06-23T18:35:55.288801+0000","last_became_active":"2023-06-23T18:35:53.133287+0000","last_became_peered":"2023-06-23T18:35:53.133287+0000","last_unstale":"2023-06-23T18:35:55.288801+0000","last_undegraded":"2023-06-23T18:35:55.288801+0000","last_fullsized":"2023-06-23T18:35:55.288801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:30:44.257929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526733+0000","last_change":"2023-06-23T18:35:53.534074+0000","last_active":"2023-06-23T18:35:54.526733+0000","last_peered":"2023-06-23T18:35:54.526733+0000","last_clean":"2023-06-23T18:35:54.526733+0000","last_became_active":"2023-06-23T18:35:53.533789+0000","last_became_peered":"2023-06-23T18:35:53.533789+0000","last_unstale":"2023-06-23T18:35:54.526733+0000","last_undegraded":"2023-06-23T18:35:54.526733+0000","last_fullsized":"2023-06-23T18:35:54.526733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:58.731924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257244+0000","last_change":"2023-06-23T18:35:53.126156+0000","last_active":"2023-06-23T18:35:55.257244+0000","last_peered":"2023-06-23T18:35:55.257244+0000","last_clean":"2023-06-23T18:35:55.257244+0000","last_became_active":"2023-06-23T18:35:53.125640+0000","last_became_peered":"2023-06-23T18:35:53.125640+0000","last_unstale":"2023-06-23T18:35:55.257244+0000","last_undegraded":"2023-06-23T18:35:55.257244+0000","last_fullsized":"2023-06-23T18:35:55.257244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:16.015103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526631+0000","last_change":"2023-06-23T18:35:53.529926+0000","last_active":"2023-06-23T18:35:54.526631+0000","last_peered":"2023-06-23T18:35:54.526631+0000","last_clean":"2023-06-23T18:35:54.526631+0000","last_became_active":"2023-06-23T18:35:53.529619+0000","last_became_peered":"2023-06-23T18:35:53.529619+0000","last_unstale":"2023-06-23T18:35:54.526631+0000","last_undegraded":"2023-06-23T18:35:54.526631+0000","last_fullsized":"2023-06-23T18:35:54.526631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:44:03.523267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526406+0000","last_change":"2023-06-23T18:35:53.134550+0000","last_active":"2023-06-23T18:35:54.526406+0000","last_peered":"2023-06-23T18:35:54.526406+0000","last_clean":"2023-06-23T18:35:54.526406+0000","last_became_active":"2023-06-23T18:35:53.132753+0000","last_became_peered":"2023-06-23T18:35:53.132753+0000","last_unstale":"2023-06-23T18:35:54.526406+0000","last_undegraded":"2023-06-23T18:35:54.526406+0000","last_fullsized":"2023-06-23T18:35:54.526406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:19:43.072740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526469+0000","last_change":"2023-06-23T18:35:53.541907+0000","last_active":"2023-06-23T18:35:54.526469+0000","last_peered":"2023-06-23T18:35:54.526469+0000","last_clean":"2023-06-23T18:35:54.526469+0000","last_became_active":"2023-06-23T18:35:53.541545+0000","last_became_peered":"2023-06-23T18:35:53.541545+0000","last_unstale":"2023-06-23T18:35:54.526469+0000","last_undegraded":"2023-06-23T18:35:54.526469+0000","last_fullsized":"2023-06-23T18:35:54.526469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:02:35.233797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288368+0000","last_change":"2023-06-23T18:35:53.130943+0000","last_active":"2023-06-23T18:35:55.288368+0000","last_peered":"2023-06-23T18:35:55.288368+0000","last_clean":"2023-06-23T18:35:55.288368+0000","last_became_active":"2023-06-23T18:35:53.130544+0000","last_became_peered":"2023-06-23T18:35:53.130544+0000","last_unstale":"2023-06-23T18:35:55.288368+0000","last_undegraded":"2023-06-23T18:35:55.288368+0000","last_fullsized":"2023-06-23T18:35:55.288368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:18.373567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527635+0000","last_change":"2023-06-23T18:35:53.535355+0000","last_active":"2023-06-23T18:35:54.527635+0000","last_peered":"2023-06-23T18:35:54.527635+0000","last_clean":"2023-06-23T18:35:54.527635+0000","last_became_active":"2023-06-23T18:35:53.535087+0000","last_became_peered":"2023-06-23T18:35:53.535087+0000","last_unstale":"2023-06-23T18:35:54.527635+0000","last_undegraded":"2023-06-23T18:35:54.527635+0000","last_fullsized":"2023-06-23T18:35:54.527635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288978+0000","last_change":"2023-06-23T18:35:53.133469+0000","last_active":"2023-06-23T18:35:55.288978+0000","last_peered":"2023-06-23T18:35:55.288978+0000","last_clean":"2023-06-23T18:35:55.288978+0000","last_became_active":"2023-06-23T18:35:53.133211+0000","last_became_peered":"2023-06-23T18:35:53.133211+0000","last_unstale":"2023-06-23T18:35:55.288978+0000","last_undegraded":"2023-06-23T18:35:55.288978+0000","last_fullsized":"2023-06-23T18:35:55.288978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:57:45.216699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526357+0000","last_change":"2023-06-23T18:35:48.920009+0000","last_active":"2023-06-23T18:35:54.526357+0000","last_peered":"2023-06-23T18:35:54.526357+0000","last_clean":"2023-06-23T18:35:54.526357+0000","last_became_active":"2023-06-23T18:35:48.919686+0000","last_became_peered":"2023-06-23T18:35:48.919686+0000","last_unstale":"2023-06-23T18:35:54.526357+0000","last_undegraded":"2023-06-23T18:35:54.526357+0000","last_fullsized":"2023-06-23T18:35:54.526357+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:57:40.062937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288250+0000","last_change":"2023-06-23T18:35:53.538294+0000","last_active":"2023-06-23T18:35:55.288250+0000","last_peered":"2023-06-23T18:35:55.288250+0000","last_clean":"2023-06-23T18:35:55.288250+0000","last_became_active":"2023-06-23T18:35:53.537996+0000","last_became_peered":"2023-06-23T18:35:53.537996+0000","last_unstale":"2023-06-23T18:35:55.288250+0000","last_undegraded":"2023-06-23T18:35:55.288250+0000","last_fullsized":"2023-06-23T18:35:55.288250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288977+0000","last_change":"2023-06-23T18:35:47.962164+0000","last_active":"2023-06-23T18:35:55.288977+0000","last_peered":"2023-06-23T18:35:55.288977+0000","last_clean":"2023-06-23T18:35:55.288977+0000","last_became_active":"2023-06-23T18:35:47.961790+0000","last_became_peered":"2023-06-23T18:35:47.961790+0000","last_unstale":"2023-06-23T18:35:55.288977+0000","last_undegraded":"2023-06-23T18:35:55.288977+0000","last_fullsized":"2023-06-23T18:35:55.288977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:25:25.504108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.944056+0000","last_change":"2023-06-23T18:35:53.126850+0000","last_active":"2023-06-23T18:35:55.944056+0000","last_peered":"2023-06-23T18:35:55.944056+0000","last_clean":"2023-06-23T18:35:55.944056+0000","last_became_active":"2023-06-23T18:35:53.126560+0000","last_became_peered":"2023-06-23T18:35:53.126560+0000","last_unstale":"2023-06-23T18:35:55.944056+0000","last_undegraded":"2023-06-23T18:35:55.944056+0000","last_fullsized":"2023-06-23T18:35:55.944056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:38.016680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526607+0000","last_change":"2023-06-23T18:35:53.531624+0000","last_active":"2023-06-23T18:35:54.526607+0000","last_peered":"2023-06-23T18:35:54.526607+0000","last_clean":"2023-06-23T18:35:54.526607+0000","last_became_active":"2023-06-23T18:35:53.531197+0000","last_became_peered":"2023-06-23T18:35:53.531197+0000","last_unstale":"2023-06-23T18:35:54.526607+0000","last_undegraded":"2023-06-23T18:35:54.526607+0000","last_fullsized":"2023-06-23T18:35:54.526607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:17.520121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527435+0000","last_change":"2023-06-23T18:35:47.965867+0000","last_active":"2023-06-23T18:35:54.527435+0000","last_peered":"2023-06-23T18:35:54.527435+0000","last_clean":"2023-06-23T18:35:54.527435+0000","last_became_active":"2023-06-23T18:35:47.965580+0000","last_became_peered":"2023-06-23T18:35:47.965580+0000","last_unstale":"2023-06-23T18:35:54.527435+0000","last_undegraded":"2023-06-23T18:35:54.527435+0000","last_fullsized":"2023-06-23T18:35:54.527435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:01.491795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793869+0000","last_change":"2023-06-23T18:35:47.968534+0000","last_active":"2023-06-23T18:35:54.793869+0000","last_peered":"2023-06-23T18:35:54.793869+0000","last_clean":"2023-06-23T18:35:54.793869+0000","last_became_active":"2023-06-23T18:35:47.967508+0000","last_became_peered":"2023-06-23T18:35:47.967508+0000","last_unstale":"2023-06-23T18:35:54.793869+0000","last_undegraded":"2023-06-23T18:35:54.793869+0000","last_fullsized":"2023-06-23T18:35:54.793869+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:07:58.101257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287523+0000","last_change":"2023-06-23T18:35:53.126632+0000","last_active":"2023-06-23T18:35:55.287523+0000","last_peered":"2023-06-23T18:35:55.287523+0000","last_clean":"2023-06-23T18:35:55.287523+0000","last_became_active":"2023-06-23T18:35:53.126336+0000","last_became_peered":"2023-06-23T18:35:53.126336+0000","last_unstale":"2023-06-23T18:35:55.287523+0000","last_undegraded":"2023-06-23T18:35:55.287523+0000","last_fullsized":"2023-06-23T18:35:55.287523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:51.656412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288240+0000","last_change":"2023-06-23T18:35:53.524885+0000","last_active":"2023-06-23T18:35:55.288240+0000","last_peered":"2023-06-23T18:35:55.288240+0000","last_clean":"2023-06-23T18:35:55.288240+0000","last_became_active":"2023-06-23T18:35:53.524630+0000","last_became_peered":"2023-06-23T18:35:53.524630+0000","last_unstale":"2023-06-23T18:35:55.288240+0000","last_undegraded":"2023-06-23T18:35:55.288240+0000","last_fullsized":"2023-06-23T18:35:55.288240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793589+0000","last_change":"2023-06-23T18:35:48.736407+0000","last_active":"2023-06-23T18:35:54.793589+0000","last_peered":"2023-06-23T18:35:54.793589+0000","last_clean":"2023-06-23T18:35:54.793589+0000","last_became_active":"2023-06-23T18:35:48.735621+0000","last_became_peered":"2023-06-23T18:35:48.735621+0000","last_unstale":"2023-06-23T18:35:54.793589+0000","last_undegraded":"2023-06-23T18:35:54.793589+0000","last_fullsized":"2023-06-23T18:35:54.793589+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:30:39.104167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527092+0000","last_change":"2023-06-23T18:35:53.131817+0000","last_active":"2023-06-23T18:35:54.527092+0000","last_peered":"2023-06-23T18:35:54.527092+0000","last_clean":"2023-06-23T18:35:54.527092+0000","last_became_active":"2023-06-23T18:35:53.131549+0000","last_became_peered":"2023-06-23T18:35:53.131549+0000","last_unstale":"2023-06-23T18:35:54.527092+0000","last_undegraded":"2023-06-23T18:35:54.527092+0000","last_fullsized":"2023-06-23T18:35:54.527092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:04:00.010842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527627+0000","last_change":"2023-06-23T18:35:53.531485+0000","last_active":"2023-06-23T18:35:54.527627+0000","last_peered":"2023-06-23T18:35:54.527627+0000","last_clean":"2023-06-23T18:35:54.527627+0000","last_became_active":"2023-06-23T18:35:53.531276+0000","last_became_peered":"2023-06-23T18:35:53.531276+0000","last_unstale":"2023-06-23T18:35:54.527627+0000","last_undegraded":"2023-06-23T18:35:54.527627+0000","last_fullsized":"2023-06-23T18:35:54.527627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:19.606579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.793552+0000","last_change":"2023-06-23T18:35:48.736449+0000","last_active":"2023-06-23T18:35:54.793552+0000","last_peered":"2023-06-23T18:35:54.793552+0000","last_clean":"2023-06-23T18:35:54.793552+0000","last_became_active":"2023-06-23T18:35:48.735664+0000","last_became_peered":"2023-06-23T18:35:48.735664+0000","last_unstale":"2023-06-23T18:35:54.793552+0000","last_undegraded":"2023-06-23T18:35:54.793552+0000","last_fullsized":"2023-06-23T18:35:54.793552+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:24.194443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288399+0000","last_change":"2023-06-23T18:35:53.127210+0000","last_active":"2023-06-23T18:35:55.288399+0000","last_peered":"2023-06-23T18:35:55.288399+0000","last_clean":"2023-06-23T18:35:55.288399+0000","last_became_active":"2023-06-23T18:35:53.126894+0000","last_became_peered":"2023-06-23T18:35:53.126894+0000","last_unstale":"2023-06-23T18:35:55.288399+0000","last_undegraded":"2023-06-23T18:35:55.288399+0000","last_fullsized":"2023-06-23T18:35:55.288399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:15.867340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257494+0000","last_change":"2023-06-23T18:35:53.539026+0000","last_active":"2023-06-23T18:35:55.257494+0000","last_peered":"2023-06-23T18:35:55.257494+0000","last_clean":"2023-06-23T18:35:55.257494+0000","last_became_active":"2023-06-23T18:35:53.538711+0000","last_became_peered":"2023-06-23T18:35:53.538711+0000","last_unstale":"2023-06-23T18:35:55.257494+0000","last_undegraded":"2023-06-23T18:35:55.257494+0000","last_fullsized":"2023-06-23T18:35:55.257494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:00.677969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526508+0000","last_change":"2023-06-23T18:35:48.735713+0000","last_active":"2023-06-23T18:35:54.526508+0000","last_peered":"2023-06-23T18:35:54.526508+0000","last_clean":"2023-06-23T18:35:54.526508+0000","last_became_active":"2023-06-23T18:35:48.735236+0000","last_became_peered":"2023-06-23T18:35:48.735236+0000","last_unstale":"2023-06-23T18:35:54.526508+0000","last_undegraded":"2023-06-23T18:35:54.526508+0000","last_fullsized":"2023-06-23T18:35:54.526508+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:22.822277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526010+0000","last_change":"2023-06-23T18:35:53.131500+0000","last_active":"2023-06-23T18:35:54.526010+0000","last_peered":"2023-06-23T18:35:54.526010+0000","last_clean":"2023-06-23T18:35:54.526010+0000","last_became_active":"2023-06-23T18:35:53.131222+0000","last_became_peered":"2023-06-23T18:35:53.131222+0000","last_unstale":"2023-06-23T18:35:54.526010+0000","last_undegraded":"2023-06-23T18:35:54.526010+0000","last_fullsized":"2023-06-23T18:35:54.526010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:38.016680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527523+0000","last_change":"2023-06-23T18:35:53.532882+0000","last_active":"2023-06-23T18:35:54.527523+0000","last_peered":"2023-06-23T18:35:54.527523+0000","last_clean":"2023-06-23T18:35:54.527523+0000","last_became_active":"2023-06-23T18:35:53.532615+0000","last_became_peered":"2023-06-23T18:35:53.532615+0000","last_unstale":"2023-06-23T18:35:54.527523+0000","last_undegraded":"2023-06-23T18:35:54.527523+0000","last_fullsized":"2023-06-23T18:35:54.527523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:33:02.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257469+0000","last_change":"2023-06-23T18:35:48.735631+0000","last_active":"2023-06-23T18:35:55.257469+0000","last_peered":"2023-06-23T18:35:55.257469+0000","last_clean":"2023-06-23T18:35:55.257469+0000","last_became_active":"2023-06-23T18:35:48.735255+0000","last_became_peered":"2023-06-23T18:35:48.735255+0000","last_unstale":"2023-06-23T18:35:55.257469+0000","last_undegraded":"2023-06-23T18:35:55.257469+0000","last_fullsized":"2023-06-23T18:35:55.257469+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:32:56.020041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527397+0000","last_change":"2023-06-23T18:35:53.132237+0000","last_active":"2023-06-23T18:35:54.527397+0000","last_peered":"2023-06-23T18:35:54.527397+0000","last_clean":"2023-06-23T18:35:54.527397+0000","last_became_active":"2023-06-23T18:35:53.131968+0000","last_became_peered":"2023-06-23T18:35:53.131968+0000","last_unstale":"2023-06-23T18:35:54.527397+0000","last_undegraded":"2023-06-23T18:35:54.527397+0000","last_fullsized":"2023-06-23T18:35:54.527397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:28:27.976038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287780+0000","last_change":"2023-06-23T18:35:53.536675+0000","last_active":"2023-06-23T18:35:55.287780+0000","last_peered":"2023-06-23T18:35:55.287780+0000","last_clean":"2023-06-23T18:35:55.287780+0000","last_became_active":"2023-06-23T18:35:53.535843+0000","last_became_peered":"2023-06-23T18:35:53.535843+0000","last_unstale":"2023-06-23T18:35:55.287780+0000","last_undegraded":"2023-06-23T18:35:55.287780+0000","last_fullsized":"2023-06-23T18:35:55.287780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:20:07.647324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257359+0000","last_change":"2023-06-23T18:35:47.970359+0000","last_active":"2023-06-23T18:35:55.257359+0000","last_peered":"2023-06-23T18:35:55.257359+0000","last_clean":"2023-06-23T18:35:55.257359+0000","last_became_active":"2023-06-23T18:35:47.970118+0000","last_became_peered":"2023-06-23T18:35:47.970118+0000","last_unstale":"2023-06-23T18:35:55.257359+0000","last_undegraded":"2023-06-23T18:35:55.257359+0000","last_fullsized":"2023-06-23T18:35:55.257359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:35.810772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527072+0000","last_change":"2023-06-23T18:35:53.132066+0000","last_active":"2023-06-23T18:35:54.527072+0000","last_peered":"2023-06-23T18:35:54.527072+0000","last_clean":"2023-06-23T18:35:54.527072+0000","last_became_active":"2023-06-23T18:35:53.131788+0000","last_became_peered":"2023-06-23T18:35:53.131788+0000","last_unstale":"2023-06-23T18:35:54.527072+0000","last_undegraded":"2023-06-23T18:35:54.527072+0000","last_fullsized":"2023-06-23T18:35:54.527072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:30.042205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527690+0000","last_change":"2023-06-23T18:35:53.536630+0000","last_active":"2023-06-23T18:35:54.527690+0000","last_peered":"2023-06-23T18:35:54.527690+0000","last_clean":"2023-06-23T18:35:54.527690+0000","last_became_active":"2023-06-23T18:35:53.536298+0000","last_became_peered":"2023-06-23T18:35:53.536298+0000","last_unstale":"2023-06-23T18:35:54.527690+0000","last_undegraded":"2023-06-23T18:35:54.527690+0000","last_fullsized":"2023-06-23T18:35:54.527690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:16.869108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287850+0000","last_change":"2023-06-23T18:35:53.126505+0000","last_active":"2023-06-23T18:35:55.287850+0000","last_peered":"2023-06-23T18:35:55.287850+0000","last_clean":"2023-06-23T18:35:55.287850+0000","last_became_active":"2023-06-23T18:35:53.126226+0000","last_became_peered":"2023-06-23T18:35:53.126226+0000","last_unstale":"2023-06-23T18:35:55.287850+0000","last_undegraded":"2023-06-23T18:35:55.287850+0000","last_fullsized":"2023-06-23T18:35:55.287850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:02:40.964533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.287939+0000","last_change":"2023-06-23T18:35:53.533402+0000","last_active":"2023-06-23T18:35:55.287939+0000","last_peered":"2023-06-23T18:35:55.287939+0000","last_clean":"2023-06-23T18:35:55.287939+0000","last_became_active":"2023-06-23T18:35:53.533163+0000","last_became_peered":"2023-06-23T18:35:53.533163+0000","last_unstale":"2023-06-23T18:35:55.287939+0000","last_undegraded":"2023-06-23T18:35:55.287939+0000","last_fullsized":"2023-06-23T18:35:55.287939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:01:29.821609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526906+0000","last_change":"2023-06-23T18:35:53.123684+0000","last_active":"2023-06-23T18:35:54.526906+0000","last_peered":"2023-06-23T18:35:54.526906+0000","last_clean":"2023-06-23T18:35:54.526906+0000","last_became_active":"2023-06-23T18:35:53.123369+0000","last_became_peered":"2023-06-23T18:35:53.123369+0000","last_unstale":"2023-06-23T18:35:54.526906+0000","last_undegraded":"2023-06-23T18:35:54.526906+0000","last_fullsized":"2023-06-23T18:35:54.526906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:18.604812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525869+0000","last_change":"2023-06-23T18:35:53.534162+0000","last_active":"2023-06-23T18:35:54.525869+0000","last_peered":"2023-06-23T18:35:54.525869+0000","last_clean":"2023-06-23T18:35:54.525869+0000","last_became_active":"2023-06-23T18:35:53.533853+0000","last_became_peered":"2023-06-23T18:35:53.533853+0000","last_unstale":"2023-06-23T18:35:54.525869+0000","last_undegraded":"2023-06-23T18:35:54.525869+0000","last_fullsized":"2023-06-23T18:35:54.525869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:30.583311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527209+0000","last_change":"2023-06-23T18:35:53.123532+0000","last_active":"2023-06-23T18:35:54.527209+0000","last_peered":"2023-06-23T18:35:54.527209+0000","last_clean":"2023-06-23T18:35:54.527209+0000","last_became_active":"2023-06-23T18:35:53.123228+0000","last_became_peered":"2023-06-23T18:35:53.123228+0000","last_unstale":"2023-06-23T18:35:54.527209+0000","last_undegraded":"2023-06-23T18:35:54.527209+0000","last_fullsized":"2023-06-23T18:35:54.527209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:29.581544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527107+0000","last_change":"2023-06-23T18:35:53.533902+0000","last_active":"2023-06-23T18:35:54.527107+0000","last_peered":"2023-06-23T18:35:54.527107+0000","last_clean":"2023-06-23T18:35:54.527107+0000","last_became_active":"2023-06-23T18:35:53.533563+0000","last_became_peered":"2023-06-23T18:35:53.533563+0000","last_unstale":"2023-06-23T18:35:54.527107+0000","last_undegraded":"2023-06-23T18:35:54.527107+0000","last_fullsized":"2023-06-23T18:35:54.527107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:55:12.340901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288866+0000","last_change":"2023-06-23T18:35:53.126195+0000","last_active":"2023-06-23T18:35:55.288866+0000","last_peered":"2023-06-23T18:35:55.288866+0000","last_clean":"2023-06-23T18:35:55.288866+0000","last_became_active":"2023-06-23T18:35:53.125886+0000","last_became_peered":"2023-06-23T18:35:53.125886+0000","last_unstale":"2023-06-23T18:35:55.288866+0000","last_undegraded":"2023-06-23T18:35:55.288866+0000","last_fullsized":"2023-06-23T18:35:55.288866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:57.730157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288616+0000","last_change":"2023-06-23T18:35:53.534747+0000","last_active":"2023-06-23T18:35:55.288616+0000","last_peered":"2023-06-23T18:35:55.288616+0000","last_clean":"2023-06-23T18:35:55.288616+0000","last_became_active":"2023-06-23T18:35:53.534495+0000","last_became_peered":"2023-06-23T18:35:53.534495+0000","last_unstale":"2023-06-23T18:35:55.288616+0000","last_undegraded":"2023-06-23T18:35:55.288616+0000","last_fullsized":"2023-06-23T18:35:55.288616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:50:30.583311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288416+0000","last_change":"2023-06-23T18:35:53.130647+0000","last_active":"2023-06-23T18:35:55.288416+0000","last_peered":"2023-06-23T18:35:55.288416+0000","last_clean":"2023-06-23T18:35:55.288416+0000","last_became_active":"2023-06-23T18:35:53.129738+0000","last_became_peered":"2023-06-23T18:35:53.129738+0000","last_unstale":"2023-06-23T18:35:55.288416+0000","last_undegraded":"2023-06-23T18:35:55.288416+0000","last_fullsized":"2023-06-23T18:35:55.288416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:52.106720+0000","last_clean_scrub_stamp":"2023-06-23T18:35:52.106720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:14:16.518353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.525676+0000","last_change":"2023-06-23T18:35:53.540648+0000","last_active":"2023-06-23T18:35:54.525676+0000","last_peered":"2023-06-23T18:35:54.525676+0000","last_clean":"2023-06-23T18:35:54.525676+0000","last_became_active":"2023-06-23T18:35:53.540380+0000","last_became_peered":"2023-06-23T18:35:53.540380+0000","last_unstale":"2023-06-23T18:35:54.525676+0000","last_undegraded":"2023-06-23T18:35:54.525676+0000","last_fullsized":"2023-06-23T18:35:54.525676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:37:19.375335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.257206+0000","last_change":"2023-06-23T18:35:53.539029+0000","last_active":"2023-06-23T18:35:55.257206+0000","last_peered":"2023-06-23T18:35:55.257206+0000","last_clean":"2023-06-23T18:35:55.257206+0000","last_became_active":"2023-06-23T18:35:53.538640+0000","last_became_peered":"2023-06-23T18:35:53.538640+0000","last_unstale":"2023-06-23T18:35:55.257206+0000","last_undegraded":"2023-06-23T18:35:55.257206+0000","last_fullsized":"2023-06-23T18:35:55.257206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:52.658179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288459+0000","last_change":"2023-06-23T18:35:53.541022+0000","last_active":"2023-06-23T18:35:55.288459+0000","last_peered":"2023-06-23T18:35:55.288459+0000","last_clean":"2023-06-23T18:35:55.288459+0000","last_became_active":"2023-06-23T18:35:53.540744+0000","last_became_peered":"2023-06-23T18:35:53.540744+0000","last_unstale":"2023-06-23T18:35:55.288459+0000","last_undegraded":"2023-06-23T18:35:55.288459+0000","last_fullsized":"2023-06-23T18:35:55.288459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:31.043972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.526923+0000","last_change":"2023-06-23T18:35:53.540406+0000","last_active":"2023-06-23T18:35:54.526923+0000","last_peered":"2023-06-23T18:35:54.526923+0000","last_clean":"2023-06-23T18:35:54.526923+0000","last_became_active":"2023-06-23T18:35:53.540125+0000","last_became_peered":"2023-06-23T18:35:53.540125+0000","last_unstale":"2023-06-23T18:35:54.526923+0000","last_undegraded":"2023-06-23T18:35:54.526923+0000","last_fullsized":"2023-06-23T18:35:54.526923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:45:17.669839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:55.288781+0000","last_change":"2023-06-23T18:35:53.542899+0000","last_active":"2023-06-23T18:35:55.288781+0000","last_peered":"2023-06-23T18:35:55.288781+0000","last_clean":"2023-06-23T18:35:55.288781+0000","last_became_active":"2023-06-23T18:35:53.541798+0000","last_became_peered":"2023-06-23T18:35:53.541798+0000","last_unstale":"2023-06-23T18:35:55.288781+0000","last_undegraded":"2023-06-23T18:35:55.288781+0000","last_fullsized":"2023-06-23T18:35:55.288781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:30.349972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-23T18:35:54.527379+0000","last_change":"2023-06-23T18:35:53.532923+0000","last_active":"2023-06-23T18:35:54.527379+0000","last_peered":"2023-06-23T18:35:54.527379+0000","last_clean":"2023-06-23T18:35:54.527379+0000","last_became_active":"2023-06-23T18:35:53.532674+0000","last_became_peered":"2023-06-23T18:35:53.532674+0000","last_unstale":"2023-06-23T18:35:54.527379+0000","last_undegraded":"2023-06-23T18:35:54.527379+0000","last_fullsized":"2023-06-23T18:35:54.527379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:53.108487+0000","last_clean_scrub_stamp":"2023-06-23T18:35:53.108487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:17.016870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":823296,"data_stored":665215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.798}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9550000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":74428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":798720,"data_stored":664796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":73649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":200704,"data_stored":73067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":208896,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1559999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":76048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":75080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:36:02.670 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-06-23T18:36:02.670 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-06-23T18:36:02.671 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-06-23T18:36:02.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-06-23T18:36:02.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.817+0000 7fd2a7b36700 1 Processor -- start 2023-06-23T18:36:02.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.817+0000 7fd2a7b36700 1 -- start start 2023-06-23T18:36:02.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.818+0000 7fd2a7b36700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd2a00cec10 0x7fd2a00cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.818+0000 7fd2a7b36700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a01438c0 0x7fd2a00cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.818+0000 7fd2a7b36700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a015de70 con 0x7fd2a00cca50 2023-06-23T18:36:02.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.818+0000 7fd2a7b36700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a015e0c0 con 0x7fd2a00cec10 2023-06-23T18:36:02.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.818+0000 7fd2a7b36700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a015e3a0 con 0x7fd2a01438c0 2023-06-23T18:36:02.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.818+0000 7fd2a50d1700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd2a00cec10 0x7fd2a00cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.818+0000 7fd2a60d3700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a01438c0 0x7fd2a00cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.818+0000 7fd2a58d2700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fd2a00cca50 0x7fd2a00ce500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:50910/0 (socket says 172.21.15.79:50910) 2023-06-23T18:36:02.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.819+0000 7fd2a58d2700 1 -- 172.21.15.79:0/1857393512 learned_addr learned my addr 172.21.15.79:0/1857393512 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:02.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.819+0000 7fd2a50d1700 1 -- 172.21.15.79:0/1857393512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a01438c0 msgr2=0x7fd2a00cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:02.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.819+0000 7fd2a50d1700 1 --2- 172.21.15.79:0/1857393512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a01438c0 0x7fd2a00cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.819+0000 7fd2a50d1700 1 -- 172.21.15.79:0/1857393512 >> v1:172.21.15.155:6789/0 conn(0x7fd2a00cca50 legacy=0x7fd2a00ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.819+0000 7fd2a50d1700 1 -- 172.21.15.79:0/1857393512 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a015e680 con 0x7fd2a00cec10 2023-06-23T18:36:02.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.819+0000 7fd2a48d0700 1 -- 172.21.15.79:0/1857393512 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11331093 0 0) 0x7fd2a015de70 con 0x7fd2a00cca50 2023-06-23T18:36:02.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.819+0000 7fd2a50d1700 1 --2- 172.21.15.79:0/1857393512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00cec10 0x7fd2a00cb180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009840 tx=0x7fd29c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=900de4ecc02bd2d1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:02.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.819+0000 7fd2a48d0700 1 -- 172.21.15.79:0/1857393512 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c011830 con 0x7fd2a00cec10 2023-06-23T18:36:02.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.819+0000 7fd2a48d0700 1 -- 172.21.15.79:0/1857393512 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c011990 con 0x7fd2a00cec10 2023-06-23T18:36:02.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.819+0000 7fd2a48d0700 1 -- 172.21.15.79:0/1857393512 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c011d20 con 0x7fd2a00cec10 2023-06-23T18:36:02.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.820+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1857393512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00cec10 msgr2=0x7fd2a00cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.820+0000 7fd2a7b36700 1 --2- 172.21.15.79:0/1857393512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00cec10 0x7fd2a00cb180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009840 tx=0x7fd29c009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:02.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.820+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1857393512 shutdown_connections 2023-06-23T18:36:02.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.820+0000 7fd2a7b36700 1 --2- 172.21.15.79:0/1857393512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a01438c0 0x7fd2a00cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.820+0000 7fd2a7b36700 1 --2- 172.21.15.79:0/1857393512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00cec10 0x7fd2a00cb180 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.820+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1857393512 >> 172.21.15.79:0/1857393512 conn(0x7fd2a0089fc0 msgr2=0x7fd2a008a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:02.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.820+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1857393512 shutdown_connections 2023-06-23T18:36:02.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.820+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1857393512 wait complete. 2023-06-23T18:36:02.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.821+0000 7fd2a7b36700 1 Processor -- start 2023-06-23T18:36:02.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.821+0000 7fd2a7b36700 1 -- start start 2023-06-23T18:36:02.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a7b36700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2a00cca50 0x7fd2a0152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a7b36700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00cec10 0x7fd2a006c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a7b36700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a01438c0 0x7fd2a00ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a7b36700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a00c9f30 con 0x7fd2a00cca50 2023-06-23T18:36:02.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a7b36700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a00c96d0 con 0x7fd2a00cec10 2023-06-23T18:36:02.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a7b36700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a00c9970 con 0x7fd2a01438c0 2023-06-23T18:36:02.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a58d2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2a00cca50 0x7fd2a0152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a58d2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2a00cca50 0x7fd2a0152eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39606/0 (socket says 172.21.15.79:39606) 2023-06-23T18:36:02.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a58d2700 1 -- 172.21.15.79:0/1693263068 learned_addr learned my addr 172.21.15.79:0/1693263068 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:02.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a60d3700 1 --2- 172.21.15.79:0/1693263068 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a01438c0 0x7fd2a00ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a50d1700 1 --2- 172.21.15.79:0/1693263068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00cec10 0x7fd2a006c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a58d2700 1 -- 172.21.15.79:0/1693263068 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a01438c0 msgr2=0x7fd2a00ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a58d2700 1 --2- 172.21.15.79:0/1693263068 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a01438c0 0x7fd2a00ae0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a58d2700 1 -- 172.21.15.79:0/1693263068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00cec10 msgr2=0x7fd2a006c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:02.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a58d2700 1 --2- 172.21.15.79:0/1693263068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00cec10 0x7fd2a006c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:02.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a58d2700 1 -- 172.21.15.79:0/1693263068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c0094a0 con 0x7fd2a00cca50 2023-06-23T18:36:02.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a58d2700 1 --2- 172.21.15.79:0/1693263068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2a00cca50 0x7fd2a0152eb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd29000c550 tx=0x7fd29000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:02.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2967fc700 1 -- 172.21.15.79:0/1693263068 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290004c80 con 0x7fd2a00cca50 2023-06-23T18:36:02.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2967fc700 1 -- 172.21.15.79:0/1693263068 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd290004e00 con 0x7fd2a00cca50 2023-06-23T18:36:02.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1693263068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a00c9c50 con 0x7fd2a00cca50 2023-06-23T18:36:02.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.822+0000 7fd2967fc700 1 -- 172.21.15.79:0/1693263068 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290007310 con 0x7fd2a00cca50 2023-06-23T18:36:02.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.823+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1693263068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a01533f0 con 0x7fd2a00cca50 2023-06-23T18:36:02.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.823+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1693263068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a015a9c0 con 0x7fd2a00cca50 2023-06-23T18:36:02.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.824+0000 7fd2967fc700 1 -- 172.21.15.79:0/1693263068 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd290007470 con 0x7fd2a00cca50 2023-06-23T18:36:02.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.824+0000 7fd2967fc700 1 --2- 172.21.15.79:0/1693263068 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd28406ace0 0x7fd28406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:02.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.824+0000 7fd2967fc700 1 -- 172.21.15.79:0/1693263068 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd290089100 con 0x7fd2a00cca50 2023-06-23T18:36:02.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.824+0000 7fd2a50d1700 1 --2- 172.21.15.79:0/1693263068 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd28406ace0 0x7fd28406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:02.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.825+0000 7fd2a50d1700 1 --2- 172.21.15.79:0/1693263068 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd28406ace0 0x7fd28406d190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0066900 tx=0x7fd29c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:02.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:02.828+0000 7fd2967fc700 1 -- 172.21.15.79:0/1693263068 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd2a015a9c0 con 0x7fd2a00cca50 2023-06-23T18:36:03.061 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.056+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1693263068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd2a015a9c0 con 0x7fd2a00cca50 2023-06-23T18:36:03.061 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.056+0000 7fd2967fc700 1 -- 172.21.15.79:0/1693263068 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd2a015a9c0 con 0x7fd2a00cca50 2023-06-23T18:36:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:03.062 INFO:teuthology.orchestra.run.smithi079.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-06-23T18:36:03.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.058+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1693263068 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd28406ace0 msgr2=0x7fd28406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.058+0000 7fd2a7b36700 1 --2- 172.21.15.79:0/1693263068 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd28406ace0 0x7fd28406d190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0066900 tx=0x7fd29c008030 comp rx=0 tx=0).stop 2023-06-23T18:36:03.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.058+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1693263068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2a00cca50 msgr2=0x7fd2a0152eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.058+0000 7fd2a7b36700 1 --2- 172.21.15.79:0/1693263068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2a00cca50 0x7fd2a0152eb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd29000c550 tx=0x7fd29000c910 comp rx=0 tx=0).stop 2023-06-23T18:36:03.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.058+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1693263068 shutdown_connections 2023-06-23T18:36:03.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.058+0000 7fd2a7b36700 1 --2- 172.21.15.79:0/1693263068 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a01438c0 0x7fd2a00ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.058+0000 7fd2a7b36700 1 --2- 172.21.15.79:0/1693263068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2a00cca50 0x7fd2a0152eb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.058+0000 7fd2a7b36700 1 --2- 172.21.15.79:0/1693263068 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd28406ace0 0x7fd28406d190 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.058+0000 7fd2a7b36700 1 --2- 172.21.15.79:0/1693263068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00cec10 0x7fd2a006c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.058+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1693263068 >> 172.21.15.79:0/1693263068 conn(0x7fd2a0089fc0 msgr2=0x7fd2a00af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:03.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.058+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1693263068 shutdown_connections 2023-06-23T18:36:03.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.058+0000 7fd2a7b36700 1 -- 172.21.15.79:0/1693263068 wait complete. 2023-06-23T18:36:03.075 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-06-23T18:36:03.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.190+0000 7f2b1572d700 1 Processor -- start 2023-06-23T18:36:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.191+0000 7f2b1572d700 1 -- start start 2023-06-23T18:36:03.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.191+0000 7f2b1572d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2b10158830 0x7f2b100c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.191+0000 7f2b1572d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2b1006a5a0 0x7f2b100c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.191+0000 7f2b1572d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b100c3f80 con 0x7f2b100c5280 2023-06-23T18:36:03.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.191+0000 7f2b1572d700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b100c4260 con 0x7f2b1006a5a0 2023-06-23T18:36:03.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.191+0000 7f2b1572d700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b100c4540 con 0x7f2b10158830 2023-06-23T18:36:03.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.192+0000 7f2b0f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2b1006a5a0 0x7f2b100c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.192+0000 7f2b0f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2b1006a5a0 0x7f2b100c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39618/0 (socket says 172.21.15.79:39618) 2023-06-23T18:36:03.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.192+0000 7f2b0f7fe700 1 -- 172.21.15.79:0/3047792214 learned_addr learned my addr 172.21.15.79:0/3047792214 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:03.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.192+0000 7f2b0effd700 1 --2- 172.21.15.79:0/3047792214 >> v2:172.21.15.155:3300/0 conn(0x7f2b10158830 0x7f2b100c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.192+0000 7f2b0f7fe700 1 -- 172.21.15.79:0/3047792214 >> v2:172.21.15.155:3300/0 conn(0x7f2b10158830 msgr2=0x7f2b100c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:03.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.192+0000 7f2b0f7fe700 1 --2- 172.21.15.79:0/3047792214 >> v2:172.21.15.155:3300/0 conn(0x7f2b10158830 0x7f2b100c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.192+0000 7f2b0f7fe700 1 -- 172.21.15.79:0/3047792214 >> v1:172.21.15.155:6789/0 conn(0x7f2b100c5280 legacy=0x7f2b10069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.192+0000 7f2b0f7fe700 1 -- 172.21.15.79:0/3047792214 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b100c4820 con 0x7f2b1006a5a0 2023-06-23T18:36:03.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.192+0000 7f2b0f7fe700 1 --2- 172.21.15.79:0/3047792214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b1006a5a0 0x7f2b100c92c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2b1006fde0 tx=0x7f2b0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916d01bf5169e092 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:03.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.192+0000 7f2b0dffb700 1 -- 172.21.15.79:0/3047792214 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0800e070 con 0x7f2b1006a5a0 2023-06-23T18:36:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.192+0000 7f2b0dffb700 1 -- 172.21.15.79:0/3047792214 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0800b9c0 con 0x7f2b1006a5a0 2023-06-23T18:36:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.192+0000 7f2b0dffb700 1 -- 172.21.15.79:0/3047792214 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0800bd70 con 0x7f2b1006a5a0 2023-06-23T18:36:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.193+0000 7f2b1572d700 1 -- 172.21.15.79:0/3047792214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b1006a5a0 msgr2=0x7f2b100c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.193+0000 7f2b1572d700 1 --2- 172.21.15.79:0/3047792214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b1006a5a0 0x7f2b100c92c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2b1006fde0 tx=0x7f2b0800b330 comp rx=0 tx=0).stop 2023-06-23T18:36:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.193+0000 7f2b1572d700 1 -- 172.21.15.79:0/3047792214 shutdown_connections 2023-06-23T18:36:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.193+0000 7f2b1572d700 1 --2- 172.21.15.79:0/3047792214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b1006a5a0 0x7f2b100c92c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.193+0000 7f2b1572d700 1 --2- 172.21.15.79:0/3047792214 >> v2:172.21.15.155:3300/0 conn(0x7f2b10158830 0x7f2b100c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.193+0000 7f2b1572d700 1 -- 172.21.15.79:0/3047792214 >> 172.21.15.79:0/3047792214 conn(0x7f2b100cf3d0 msgr2=0x7f2b100cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:03.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.193+0000 7f2b1572d700 1 -- 172.21.15.79:0/3047792214 shutdown_connections 2023-06-23T18:36:03.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.193+0000 7f2b1572d700 1 -- 172.21.15.79:0/3047792214 wait complete. 2023-06-23T18:36:03.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.194+0000 7f2b1572d700 1 Processor -- start 2023-06-23T18:36:03.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.194+0000 7f2b1572d700 1 -- start start 2023-06-23T18:36:03.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b1572d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b1006a5a0 0x7f2b10156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b1572d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100c5280 0x7f2b101444c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b1572d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b10158830 0x7f2b100b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b1572d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b10155a80 con 0x7f2b100c5280 2023-06-23T18:36:03.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b1572d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b10144a00 con 0x7f2b10158830 2023-06-23T18:36:03.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b1572d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b10155c90 con 0x7f2b1006a5a0 2023-06-23T18:36:03.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b0e7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100c5280 0x7f2b101444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b0e7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100c5280 0x7f2b101444c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39620/0 (socket says 172.21.15.79:39620) 2023-06-23T18:36:03.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b0e7fc700 1 -- 172.21.15.79:0/568029076 learned_addr learned my addr 172.21.15.79:0/568029076 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:03.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b0f7fe700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b10158830 0x7f2b100b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b0effd700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b1006a5a0 0x7f2b10156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b0e7fc700 1 -- 172.21.15.79:0/568029076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b1006a5a0 msgr2=0x7f2b10156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b0e7fc700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b1006a5a0 0x7f2b10156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b0e7fc700 1 -- 172.21.15.79:0/568029076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b10158830 msgr2=0x7f2b100b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b0e7fc700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b10158830 0x7f2b100b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b0e7fc700 1 -- 172.21.15.79:0/568029076 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b08009b30 con 0x7f2b100c5280 2023-06-23T18:36:03.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b0f7fe700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b10158830 0x7f2b100b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:03.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.195+0000 7f2b0e7fc700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100c5280 0x7f2b101444c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2b0000c4e0 tx=0x7f2b0000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:03.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.196+0000 7f2af77fe700 1 -- 172.21.15.79:0/568029076 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b000052d0 con 0x7f2b100c5280 2023-06-23T18:36:03.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.196+0000 7f2af77fe700 1 -- 172.21.15.79:0/568029076 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b00005430 con 0x7f2b100c5280 2023-06-23T18:36:03.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.196+0000 7f2b1572d700 1 -- 172.21.15.79:0/568029076 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b10155f70 con 0x7f2b100c5280 2023-06-23T18:36:03.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.196+0000 7f2af77fe700 1 -- 172.21.15.79:0/568029076 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b000038c0 con 0x7f2b100c5280 2023-06-23T18:36:03.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.196+0000 7f2b1572d700 1 -- 172.21.15.79:0/568029076 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b10156530 con 0x7f2b100c5280 2023-06-23T18:36:03.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.196+0000 7f2b1572d700 1 -- 172.21.15.79:0/568029076 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b100c39b0 con 0x7f2b100c5280 2023-06-23T18:36:03.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.197+0000 7f2af77fe700 1 -- 172.21.15.79:0/568029076 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2b00003a20 con 0x7f2b100c5280 2023-06-23T18:36:03.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.197+0000 7f2af77fe700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2afc06ace0 0x7f2afc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.197+0000 7f2af77fe700 1 -- 172.21.15.79:0/568029076 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b00088960 con 0x7f2b100c5280 2023-06-23T18:36:03.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.197+0000 7f2b0effd700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2afc06ace0 0x7f2afc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.198+0000 7f2b0effd700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2afc06ace0 0x7f2afc06d190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2b100b1400 tx=0x7f2b04008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:03.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.202+0000 7f2af77fe700 1 -- 172.21.15.79:0/568029076 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2b00053a10 con 0x7f2b100c5280 2023-06-23T18:36:03.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.388+0000 7f2b1572d700 1 -- 172.21.15.79:0/568029076 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b100c89c0 con 0x7f2b100c5280 2023-06-23T18:36:03.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.389+0000 7f2af77fe700 1 -- 172.21.15.79:0/568029076 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7f2b00057030 con 0x7f2b100c5280 2023-06-23T18:36:03.394 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:03.395 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:03.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.391+0000 7f2b1572d700 1 -- 172.21.15.79:0/568029076 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2afc06ace0 msgr2=0x7f2afc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.391+0000 7f2b1572d700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2afc06ace0 0x7f2afc06d190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2b100b1400 tx=0x7f2b04008040 comp rx=0 tx=0).stop 2023-06-23T18:36:03.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.391+0000 7f2b1572d700 1 -- 172.21.15.79:0/568029076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100c5280 msgr2=0x7f2b101444c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.391+0000 7f2b1572d700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100c5280 0x7f2b101444c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2b0000c4e0 tx=0x7f2b0000c8a0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.391+0000 7f2b1572d700 1 -- 172.21.15.79:0/568029076 shutdown_connections 2023-06-23T18:36:03.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.391+0000 7f2b1572d700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2afc06ace0 0x7f2afc06d190 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.391+0000 7f2b1572d700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b10158830 0x7f2b100b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.391+0000 7f2b1572d700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b1006a5a0 0x7f2b10156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.391+0000 7f2b1572d700 1 --2- 172.21.15.79:0/568029076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100c5280 0x7f2b101444c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.391+0000 7f2b1572d700 1 -- 172.21.15.79:0/568029076 >> 172.21.15.79:0/568029076 conn(0x7f2b100cf3d0 msgr2=0x7f2b10157470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:03.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.391+0000 7f2b1572d700 1 -- 172.21.15.79:0/568029076 shutdown_connections 2023-06-23T18:36:03.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.391+0000 7f2b1572d700 1 -- 172.21.15.79:0/568029076 wait complete. 2023-06-23T18:36:03.408 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:03.409 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:03.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.526+0000 7f3a1295b700 1 Processor -- start 2023-06-23T18:36:03.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.526+0000 7f3a1295b700 1 -- start start 2023-06-23T18:36:03.534 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.527+0000 7f3a1295b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3a0c1569c0 0x7f3a0c1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.534 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.527+0000 7f3a1295b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a0c0cd440 0x7f3a0c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.534 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.527+0000 7f3a1295b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3a0c15d580 0x7f3a0c06a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.534 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.527+0000 7f3a1295b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0c0cf570 con 0x7f3a0c1569c0 2023-06-23T18:36:03.534 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.527+0000 7f3a1295b700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0c15c8b0 con 0x7f3a0c15d580 2023-06-23T18:36:03.534 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.527+0000 7f3a1295b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c160df0 con 0x7f3a0c0cd440 2023-06-23T18:36:03.534 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.527+0000 7f3a0bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3a0c1569c0 0x7f3a0c1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.527+0000 7f3a0bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3a0c1569c0 0x7f3a0c1608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39626/0 (socket says 172.21.15.79:39626) 2023-06-23T18:36:03.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.527+0000 7f3a0bfff700 1 -- 172.21.15.79:0/2770888665 learned_addr learned my addr 172.21.15.79:0/2770888665 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:03.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.528+0000 7f3a10ef8700 1 --2- 172.21.15.79:0/2770888665 >> v2:172.21.15.155:3300/0 conn(0x7f3a0c15d580 0x7f3a0c06a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.528+0000 7f3a0b7fe700 1 --2- 172.21.15.79:0/2770888665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a0c0cd440 0x7f3a0c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.528+0000 7f3a0bfff700 1 -- 172.21.15.79:0/2770888665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a0c0cd440 msgr2=0x7f3a0c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:03.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.528+0000 7f3a0bfff700 1 --2- 172.21.15.79:0/2770888665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a0c0cd440 0x7f3a0c069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.528+0000 7f3a0bfff700 1 -- 172.21.15.79:0/2770888665 >> v2:172.21.15.155:3300/0 conn(0x7f3a0c15d580 msgr2=0x7f3a0c06a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:03.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.528+0000 7f3a0bfff700 1 --2- 172.21.15.79:0/2770888665 >> v2:172.21.15.155:3300/0 conn(0x7f3a0c15d580 0x7f3a0c06a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.528+0000 7f3a0bfff700 1 -- 172.21.15.79:0/2770888665 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c0cdc90 con 0x7f3a0c1569c0 2023-06-23T18:36:03.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.528+0000 7f3a0b7fe700 1 --2- 172.21.15.79:0/2770888665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a0c0cd440 0x7f3a0c069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:03.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.528+0000 7f3a0bfff700 1 --2- 172.21.15.79:0/2770888665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a0c1569c0 0x7f3a0c1608b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0cc2a0 tx=0x7f3a00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868c1a3915ed1508 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:03.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.528+0000 7f3a0affd700 1 -- 172.21.15.79:0/2770888665 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0000b870 con 0x7f3a0c1569c0 2023-06-23T18:36:03.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.528+0000 7f3a0affd700 1 -- 172.21.15.79:0/2770888665 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0000b9d0 con 0x7f3a0c1569c0 2023-06-23T18:36:03.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.528+0000 7f3a0affd700 1 -- 172.21.15.79:0/2770888665 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0000bd60 con 0x7f3a0c1569c0 2023-06-23T18:36:03.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.528+0000 7f3a1295b700 1 -- 172.21.15.79:0/2770888665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a0c1569c0 msgr2=0x7f3a0c1608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.528+0000 7f3a1295b700 1 --2- 172.21.15.79:0/2770888665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a0c1569c0 0x7f3a0c1608b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0cc2a0 tx=0x7f3a00009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.529+0000 7f3a1295b700 1 -- 172.21.15.79:0/2770888665 shutdown_connections 2023-06-23T18:36:03.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.529+0000 7f3a1295b700 1 --2- 172.21.15.79:0/2770888665 >> v2:172.21.15.155:3300/0 conn(0x7f3a0c15d580 0x7f3a0c06a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.529+0000 7f3a1295b700 1 --2- 172.21.15.79:0/2770888665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a0c1569c0 0x7f3a0c1608b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.529+0000 7f3a1295b700 1 --2- 172.21.15.79:0/2770888665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a0c0cd440 0x7f3a0c069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.529+0000 7f3a1295b700 1 -- 172.21.15.79:0/2770888665 >> 172.21.15.79:0/2770888665 conn(0x7f3a0c06afa0 msgr2=0x7f3a0c06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:03.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.529+0000 7f3a1295b700 1 -- 172.21.15.79:0/2770888665 shutdown_connections 2023-06-23T18:36:03.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.529+0000 7f3a1295b700 1 -- 172.21.15.79:0/2770888665 wait complete. 2023-06-23T18:36:03.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.530+0000 7f3a1295b700 1 Processor -- start 2023-06-23T18:36:03.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.530+0000 7f3a1295b700 1 -- start start 2023-06-23T18:36:03.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a1295b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a0c0cd440 0x7f3a0c0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a1295b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a0c1569c0 0x7f3a0c1f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a1295b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a0c15d580 0x7f3a0c0ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a1295b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c0cbf70 con 0x7f3a0c1569c0 2023-06-23T18:36:03.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a1295b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c1f2d40 con 0x7f3a0c0cd440 2023-06-23T18:36:03.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a1295b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c0abc50 con 0x7f3a0c15d580 2023-06-23T18:36:03.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a0b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a0c1569c0 0x7f3a0c1f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a0b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a0c1569c0 0x7f3a0c1f2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39632/0 (socket says 172.21.15.79:39632) 2023-06-23T18:36:03.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a0b7fe700 1 -- 172.21.15.79:0/1072395290 learned_addr learned my addr 172.21.15.79:0/1072395290 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:03.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a0bfff700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a0c0cd440 0x7f3a0c0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a10ef8700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a0c15d580 0x7f3a0c0ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a0b7fe700 1 -- 172.21.15.79:0/1072395290 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a0c15d580 msgr2=0x7f3a0c0ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a0b7fe700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a0c15d580 0x7f3a0c0ad800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a0b7fe700 1 -- 172.21.15.79:0/1072395290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a0c0cd440 msgr2=0x7f3a0c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a0b7fe700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a0c0cd440 0x7f3a0c0cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a0b7fe700 1 -- 172.21.15.79:0/1072395290 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a00009550 con 0x7f3a0c1569c0 2023-06-23T18:36:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a10ef8700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a0c15d580 0x7f3a0c0ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a0b7fe700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a0c1569c0 0x7f3a0c1f2800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00c750 tx=0x7f39fc00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a08ff9700 1 -- 172.21.15.79:0/1072395290 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc011800 con 0x7f3a0c1569c0 2023-06-23T18:36:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a08ff9700 1 -- 172.21.15.79:0/1072395290 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc011960 con 0x7f3a0c1569c0 2023-06-23T18:36:03.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a08ff9700 1 -- 172.21.15.79:0/1072395290 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc011cd0 con 0x7f3a0c1569c0 2023-06-23T18:36:03.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.531+0000 7f3a1295b700 1 -- 172.21.15.79:0/1072395290 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0c0abf30 con 0x7f3a0c1569c0 2023-06-23T18:36:03.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.532+0000 7f3a1295b700 1 -- 172.21.15.79:0/1072395290 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0c0ac4f0 con 0x7f3a0c1569c0 2023-06-23T18:36:03.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.532+0000 7f3a1295b700 1 -- 172.21.15.79:0/1072395290 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0c0b26a0 con 0x7f3a0c1569c0 2023-06-23T18:36:03.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.533+0000 7f3a08ff9700 1 -- 172.21.15.79:0/1072395290 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f39fc021350 con 0x7f3a0c1569c0 2023-06-23T18:36:03.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.533+0000 7f3a08ff9700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f39f806ace0 0x7f39f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.533+0000 7f3a08ff9700 1 -- 172.21.15.79:0/1072395290 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39fc089480 con 0x7f3a0c1569c0 2023-06-23T18:36:03.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.533+0000 7f3a0bfff700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f39f806ace0 0x7f39f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.544 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.533+0000 7f3a0bfff700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f39f806ace0 0x7f39f806d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3a00000c00 tx=0x7f3a00008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:03.544 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.537+0000 7f3a08ff9700 1 -- 172.21.15.79:0/1072395290 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f39fc054530 con 0x7f3a0c1569c0 2023-06-23T18:36:03.728 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.723+0000 7f3a1295b700 1 -- 172.21.15.79:0/1072395290 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3a0c0672c0 con 0x7f3a0c1569c0 2023-06-23T18:36:03.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.724+0000 7f3a08ff9700 1 -- 172.21.15.79:0/1072395290 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f39fc057b50 con 0x7f3a0c1569c0 2023-06-23T18:36:03.729 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:03.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.726+0000 7f3a1295b700 1 -- 172.21.15.79:0/1072395290 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f39f806ace0 msgr2=0x7f39f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.726+0000 7f3a1295b700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f39f806ace0 0x7f39f806d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3a00000c00 tx=0x7f3a00008030 comp rx=0 tx=0).stop 2023-06-23T18:36:03.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.726+0000 7f3a1295b700 1 -- 172.21.15.79:0/1072395290 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a0c1569c0 msgr2=0x7f3a0c1f2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.726+0000 7f3a1295b700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a0c1569c0 0x7f3a0c1f2800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00c750 tx=0x7f39fc00cb10 comp rx=0 tx=0).stop 2023-06-23T18:36:03.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.726+0000 7f3a1295b700 1 -- 172.21.15.79:0/1072395290 shutdown_connections 2023-06-23T18:36:03.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.726+0000 7f3a1295b700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a0c15d580 0x7f3a0c0ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.726+0000 7f3a1295b700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f39f806ace0 0x7f39f806d190 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.726+0000 7f3a1295b700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a0c0cd440 0x7f3a0c0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.726+0000 7f3a1295b700 1 --2- 172.21.15.79:0/1072395290 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a0c1569c0 0x7f3a0c1f2800 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.726+0000 7f3a1295b700 1 -- 172.21.15.79:0/1072395290 >> 172.21.15.79:0/1072395290 conn(0x7f3a0c06afa0 msgr2=0x7f3a0c06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:03.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.726+0000 7f3a1295b700 1 -- 172.21.15.79:0/1072395290 shutdown_connections 2023-06-23T18:36:03.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.726+0000 7f3a1295b700 1 -- 172.21.15.79:0/1072395290 wait complete. 2023-06-23T18:36:03.743 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:03.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.861+0000 7f656c6b5700 1 Processor -- start 2023-06-23T18:36:03.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.861+0000 7f656c6b5700 1 -- start start 2023-06-23T18:36:03.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.862+0000 7f656c6b5700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6564156ae0 0x7f656415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.862+0000 7f656c6b5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f65640cc4b0 0x7f6564071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.862+0000 7f656c6b5700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656415daf0 con 0x7f65640cc060 2023-06-23T18:36:03.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.862+0000 7f656c6b5700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656415a560 con 0x7f65640cc4b0 2023-06-23T18:36:03.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.862+0000 7f656c6b5700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6564159950 con 0x7f6564156ae0 2023-06-23T18:36:03.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.862+0000 7f656ac52700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f65640cc4b0 0x7f6564071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.862+0000 7f6569c50700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f65640cc060 0x7f6564071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42680/0 (socket says 172.21.15.79:42680) 2023-06-23T18:36:03.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.862+0000 7f656ac52700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f65640cc4b0 0x7f6564071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39642/0 (socket says 172.21.15.79:39642) 2023-06-23T18:36:03.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.862+0000 7f6569c50700 1 -- 172.21.15.79:0/4260189200 learned_addr learned my addr 172.21.15.79:0/4260189200 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:03.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.862+0000 7f656a451700 1 --2- 172.21.15.79:0/4260189200 >> v2:172.21.15.155:3300/0 conn(0x7f6564156ae0 0x7f656415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656ac52700 1 -- 172.21.15.79:0/4260189200 >> v2:172.21.15.155:3300/0 conn(0x7f6564156ae0 msgr2=0x7f656415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:03.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656ac52700 1 --2- 172.21.15.79:0/4260189200 >> v2:172.21.15.155:3300/0 conn(0x7f6564156ae0 0x7f656415d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656ac52700 1 -- 172.21.15.79:0/4260189200 >> v1:172.21.15.79:6789/0 conn(0x7f65640cc060 legacy=0x7f6564071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656ac52700 1 -- 172.21.15.79:0/4260189200 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6564159c30 con 0x7f65640cc4b0 2023-06-23T18:36:03.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656ac52700 1 --2- 172.21.15.79:0/4260189200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65640cc4b0 0x7f6564071e20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f65640d0910 tx=0x7f655c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab4a758a9dd54f6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:03.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656944f700 1 -- 172.21.15.79:0/4260189200 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c00e070 con 0x7f65640cc4b0 2023-06-23T18:36:03.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656944f700 1 -- 172.21.15.79:0/4260189200 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c00bbd0 con 0x7f65640cc4b0 2023-06-23T18:36:03.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656944f700 1 -- 172.21.15.79:0/4260189200 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c012430 con 0x7f65640cc4b0 2023-06-23T18:36:03.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656c6b5700 1 -- 172.21.15.79:0/4260189200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65640cc4b0 msgr2=0x7f6564071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656c6b5700 1 --2- 172.21.15.79:0/4260189200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65640cc4b0 0x7f6564071e20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f65640d0910 tx=0x7f655c00b330 comp rx=0 tx=0).stop 2023-06-23T18:36:03.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656c6b5700 1 -- 172.21.15.79:0/4260189200 shutdown_connections 2023-06-23T18:36:03.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656c6b5700 1 --2- 172.21.15.79:0/4260189200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65640cc4b0 0x7f6564071e20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656c6b5700 1 --2- 172.21.15.79:0/4260189200 >> v2:172.21.15.155:3300/0 conn(0x7f6564156ae0 0x7f656415d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656c6b5700 1 -- 172.21.15.79:0/4260189200 >> 172.21.15.79:0/4260189200 conn(0x7f656406afa0 msgr2=0x7f656406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:03.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656c6b5700 1 -- 172.21.15.79:0/4260189200 shutdown_connections 2023-06-23T18:36:03.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.863+0000 7f656c6b5700 1 -- 172.21.15.79:0/4260189200 wait complete. 2023-06-23T18:36:03.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.865+0000 7f656c6b5700 1 Processor -- start 2023-06-23T18:36:03.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.865+0000 7f656c6b5700 1 -- start start 2023-06-23T18:36:03.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.865+0000 7f656c6b5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65640cc060 0x7f65640c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.865+0000 7f656c6b5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65640cc4b0 0x7f656415a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.865+0000 7f656c6b5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6564156ae0 0x7f65640b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.865+0000 7f656c6b5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65640c59d0 con 0x7f65640cc4b0 2023-06-23T18:36:03.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.865+0000 7f656c6b5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65640c5cb0 con 0x7f6564156ae0 2023-06-23T18:36:03.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.865+0000 7f656c6b5700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65640c5f90 con 0x7f65640cc060 2023-06-23T18:36:03.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.865+0000 7f6569c50700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65640cc4b0 0x7f656415a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.865+0000 7f6569c50700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65640cc4b0 0x7f656415a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39646/0 (socket says 172.21.15.79:39646) 2023-06-23T18:36:03.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.865+0000 7f6569c50700 1 -- 172.21.15.79:0/2218260997 learned_addr learned my addr 172.21.15.79:0/2218260997 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:03.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.866+0000 7f656a451700 1 --2- 172.21.15.79:0/2218260997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65640cc060 0x7f65640c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.866+0000 7f656ac52700 1 --2- 172.21.15.79:0/2218260997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6564156ae0 0x7f65640b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.866+0000 7f6569c50700 1 -- 172.21.15.79:0/2218260997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65640cc060 msgr2=0x7f65640c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.866+0000 7f6569c50700 1 --2- 172.21.15.79:0/2218260997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65640cc060 0x7f65640c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.866+0000 7f6569c50700 1 -- 172.21.15.79:0/2218260997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6564156ae0 msgr2=0x7f65640b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:03.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.866+0000 7f6569c50700 1 --2- 172.21.15.79:0/2218260997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6564156ae0 0x7f65640b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:03.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.866+0000 7f6569c50700 1 -- 172.21.15.79:0/2218260997 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c009a70 con 0x7f65640cc4b0 2023-06-23T18:36:03.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.866+0000 7f6569c50700 1 --2- 172.21.15.79:0/2218260997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65640cc4b0 0x7f656415a940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f656000c520 tx=0x7f656000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:03.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.866+0000 7f655affd700 1 -- 172.21.15.79:0/2218260997 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560005d70 con 0x7f65640cc4b0 2023-06-23T18:36:03.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.866+0000 7f655affd700 1 -- 172.21.15.79:0/2218260997 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65600070d0 con 0x7f65640cc4b0 2023-06-23T18:36:03.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.866+0000 7f656c6b5700 1 -- 172.21.15.79:0/2218260997 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65640c69e0 con 0x7f65640cc4b0 2023-06-23T18:36:03.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.866+0000 7f655affd700 1 -- 172.21.15.79:0/2218260997 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560007440 con 0x7f65640cc4b0 2023-06-23T18:36:03.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.866+0000 7f656c6b5700 1 -- 172.21.15.79:0/2218260997 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65640b26a0 con 0x7f65640cc4b0 2023-06-23T18:36:03.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.867+0000 7f656c6b5700 1 -- 172.21.15.79:0/2218260997 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65640c6f00 con 0x7f65640cc4b0 2023-06-23T18:36:03.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.868+0000 7f655affd700 1 -- 172.21.15.79:0/2218260997 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f65600075a0 con 0x7f65640cc4b0 2023-06-23T18:36:03.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.868+0000 7f655affd700 1 --2- 172.21.15.79:0/2218260997 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f654806ace0 0x7f654806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:03.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.868+0000 7f656a451700 1 --2- 172.21.15.79:0/2218260997 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f654806ace0 0x7f654806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:03.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.868+0000 7f655affd700 1 -- 172.21.15.79:0/2218260997 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f65600887c0 con 0x7f65640cc4b0 2023-06-23T18:36:03.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.868+0000 7f656a451700 1 --2- 172.21.15.79:0/2218260997 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f654806ace0 0x7f654806d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6554006b20 tx=0x7f6554008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:03.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:03.872+0000 7f655affd700 1 -- 172.21.15.79:0/2218260997 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f65640c6f00 con 0x7f65640cc4b0 2023-06-23T18:36:04.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.060+0000 7f656c6b5700 1 -- 172.21.15.79:0/2218260997 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65640c6f00 con 0x7f65640cc4b0 2023-06-23T18:36:04.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.060+0000 7f655affd700 1 -- 172.21.15.79:0/2218260997 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65640c6f00 con 0x7f65640cc4b0 2023-06-23T18:36:04.065 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:04.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.062+0000 7f656c6b5700 1 -- 172.21.15.79:0/2218260997 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f654806ace0 msgr2=0x7f654806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.062+0000 7f656c6b5700 1 --2- 172.21.15.79:0/2218260997 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f654806ace0 0x7f654806d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6554006b20 tx=0x7f6554008040 comp rx=0 tx=0).stop 2023-06-23T18:36:04.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.062+0000 7f656c6b5700 1 -- 172.21.15.79:0/2218260997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65640cc4b0 msgr2=0x7f656415a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.062+0000 7f656c6b5700 1 --2- 172.21.15.79:0/2218260997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65640cc4b0 0x7f656415a940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f656000c520 tx=0x7f656000c830 comp rx=0 tx=0).stop 2023-06-23T18:36:04.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.062+0000 7f656c6b5700 1 -- 172.21.15.79:0/2218260997 shutdown_connections 2023-06-23T18:36:04.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.062+0000 7f656c6b5700 1 --2- 172.21.15.79:0/2218260997 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f654806ace0 0x7f654806d190 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.062+0000 7f656c6b5700 1 --2- 172.21.15.79:0/2218260997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6564156ae0 0x7f65640b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.062+0000 7f656c6b5700 1 --2- 172.21.15.79:0/2218260997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65640cc060 0x7f65640c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.062+0000 7f656c6b5700 1 --2- 172.21.15.79:0/2218260997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65640cc4b0 0x7f656415a940 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.062+0000 7f656c6b5700 1 -- 172.21.15.79:0/2218260997 >> 172.21.15.79:0/2218260997 conn(0x7f656406afa0 msgr2=0x7f656406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:04.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.062+0000 7f656c6b5700 1 -- 172.21.15.79:0/2218260997 shutdown_connections 2023-06-23T18:36:04.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.062+0000 7f656c6b5700 1 -- 172.21.15.79:0/2218260997 wait complete. 2023-06-23T18:36:04.079 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:04.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.196+0000 7fb454c98700 1 Processor -- start 2023-06-23T18:36:04.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.196+0000 7fb454c98700 1 -- start start 2023-06-23T18:36:04.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.197+0000 7fb454c98700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb45015be70 0x7fb450159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.197+0000 7fb454c98700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb450158db0 0x7fb45015c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.197+0000 7fb454c98700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb450159220 con 0x7fb4500cd040 2023-06-23T18:36:04.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.197+0000 7fb454c98700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb45015c240 con 0x7fb450158db0 2023-06-23T18:36:04.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.197+0000 7fb454c98700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45015d3c0 con 0x7fb45015be70 2023-06-23T18:36:04.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.197+0000 7fb44e59c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb45015be70 0x7fb450159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.197+0000 7fb44dd9b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb450158db0 0x7fb45015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.197+0000 7fb44dd9b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb450158db0 0x7fb45015c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:45880/0 (socket says 172.21.15.79:45880) 2023-06-23T18:36:04.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.197+0000 7fb44e59c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb45015be70 0x7fb450159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:60654/0 (socket says 172.21.15.79:60654) 2023-06-23T18:36:04.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.197+0000 7fb44ed9d700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fb4500cd040 0x7fb45006a9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:50966/0 (socket says 172.21.15.79:50966) 2023-06-23T18:36:04.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.197+0000 7fb44dd9b700 1 -- 172.21.15.79:0/2132957859 learned_addr learned my addr 172.21.15.79:0/2132957859 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:04.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.198+0000 7fb44dd9b700 1 -- 172.21.15.79:0/2132957859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb45015be70 msgr2=0x7fb450159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:04.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.198+0000 7fb44dd9b700 1 --2- 172.21.15.79:0/2132957859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb45015be70 0x7fb450159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.198+0000 7fb44dd9b700 1 -- 172.21.15.79:0/2132957859 >> v1:172.21.15.155:6789/0 conn(0x7fb4500cd040 legacy=0x7fb45006a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.198+0000 7fb44dd9b700 1 -- 172.21.15.79:0/2132957859 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45006a0b0 con 0x7fb450158db0 2023-06-23T18:36:04.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.198+0000 7fb44dd9b700 1 --2- 172.21.15.79:0/2132957859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb450158db0 0x7fb45015c420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb4400098f0 tx=0x7fb440009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17601d2cf0b6cfa3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:04.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.198+0000 7fb44d59a700 1 -- 172.21.15.79:0/2132957859 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440011830 con 0x7fb450158db0 2023-06-23T18:36:04.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.198+0000 7fb44d59a700 1 -- 172.21.15.79:0/2132957859 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb440011990 con 0x7fb450158db0 2023-06-23T18:36:04.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.198+0000 7fb44d59a700 1 -- 172.21.15.79:0/2132957859 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440011d20 con 0x7fb450158db0 2023-06-23T18:36:04.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.199+0000 7fb454c98700 1 -- 172.21.15.79:0/2132957859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb450158db0 msgr2=0x7fb45015c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.199+0000 7fb454c98700 1 --2- 172.21.15.79:0/2132957859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb450158db0 0x7fb45015c420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb4400098f0 tx=0x7fb440009c00 comp rx=0 tx=0).stop 2023-06-23T18:36:04.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.199+0000 7fb454c98700 1 -- 172.21.15.79:0/2132957859 shutdown_connections 2023-06-23T18:36:04.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.199+0000 7fb454c98700 1 --2- 172.21.15.79:0/2132957859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb45015be70 0x7fb450159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.199+0000 7fb454c98700 1 --2- 172.21.15.79:0/2132957859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb450158db0 0x7fb45015c420 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.199+0000 7fb454c98700 1 -- 172.21.15.79:0/2132957859 >> 172.21.15.79:0/2132957859 conn(0x7fb4500cfc60 msgr2=0x7fb4500b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:04.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.199+0000 7fb454c98700 1 -- 172.21.15.79:0/2132957859 shutdown_connections 2023-06-23T18:36:04.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.199+0000 7fb454c98700 1 -- 172.21.15.79:0/2132957859 wait complete. 2023-06-23T18:36:04.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.200+0000 7fb454c98700 1 Processor -- start 2023-06-23T18:36:04.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.200+0000 7fb454c98700 1 -- start start 2023-06-23T18:36:04.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb454c98700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4500cd040 0x7fb4500ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb454c98700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb450158db0 0x7fb4501f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb454c98700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb45015be70 0x7fb45006ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb454c98700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4500b1fa0 con 0x7fb450158db0 2023-06-23T18:36:04.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb454c98700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4501f31a0 con 0x7fb4500cd040 2023-06-23T18:36:04.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb454c98700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4500ad870 con 0x7fb45015be70 2023-06-23T18:36:04.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb44dd9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb450158db0 0x7fb4501f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb44dd9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb450158db0 0x7fb4501f2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39662/0 (socket says 172.21.15.79:39662) 2023-06-23T18:36:04.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb44dd9b700 1 -- 172.21.15.79:0/2123616510 learned_addr learned my addr 172.21.15.79:0/2123616510 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:04.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb44ed9d700 1 --2- 172.21.15.79:0/2123616510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb45015be70 0x7fb45006ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb44e59c700 1 --2- 172.21.15.79:0/2123616510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4500cd040 0x7fb4500ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb44dd9b700 1 -- 172.21.15.79:0/2123616510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb45015be70 msgr2=0x7fb45006ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb44dd9b700 1 --2- 172.21.15.79:0/2123616510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb45015be70 0x7fb45006ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb44dd9b700 1 -- 172.21.15.79:0/2123616510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4500cd040 msgr2=0x7fb4500ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb44dd9b700 1 --2- 172.21.15.79:0/2123616510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4500cd040 0x7fb4500ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb44dd9b700 1 -- 172.21.15.79:0/2123616510 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb440009550 con 0x7fb450158db0 2023-06-23T18:36:04.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.201+0000 7fb44dd9b700 1 --2- 172.21.15.79:0/2123616510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb450158db0 0x7fb4501f2c60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb440009c40 tx=0x7fb440004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:04.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.202+0000 7fb43effd700 1 -- 172.21.15.79:0/2123616510 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440004510 con 0x7fb450158db0 2023-06-23T18:36:04.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.202+0000 7fb43effd700 1 -- 172.21.15.79:0/2123616510 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44001e070 con 0x7fb450158db0 2023-06-23T18:36:04.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.202+0000 7fb454c98700 1 -- 172.21.15.79:0/2123616510 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4500adb50 con 0x7fb450158db0 2023-06-23T18:36:04.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.202+0000 7fb43effd700 1 -- 172.21.15.79:0/2123616510 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400038e0 con 0x7fb450158db0 2023-06-23T18:36:04.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.202+0000 7fb454c98700 1 -- 172.21.15.79:0/2123616510 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4500ae110 con 0x7fb450158db0 2023-06-23T18:36:04.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.202+0000 7fb454c98700 1 -- 172.21.15.79:0/2123616510 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4500ce210 con 0x7fb450158db0 2023-06-23T18:36:04.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.203+0000 7fb43effd700 1 -- 172.21.15.79:0/2123616510 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb440003a40 con 0x7fb450158db0 2023-06-23T18:36:04.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.204+0000 7fb43effd700 1 --2- 172.21.15.79:0/2123616510 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb43006ace0 0x7fb43006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.204+0000 7fb44e59c700 1 --2- 172.21.15.79:0/2123616510 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb43006ace0 0x7fb43006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.204+0000 7fb43effd700 1 -- 172.21.15.79:0/2123616510 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb44008a340 con 0x7fb450158db0 2023-06-23T18:36:04.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.204+0000 7fb44e59c700 1 --2- 172.21.15.79:0/2123616510 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb43006ace0 0x7fb43006d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb44400ace0 tx=0x7fb44400a320 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:04.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.208+0000 7fb43effd700 1 -- 172.21.15.79:0/2123616510 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb440055400 con 0x7fb450158db0 2023-06-23T18:36:04.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.396+0000 7fb454c98700 1 -- 172.21.15.79:0/2123616510 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4500ae300 con 0x7fb450158db0 2023-06-23T18:36:04.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.397+0000 7fb43effd700 1 -- 172.21.15.79:0/2123616510 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb440058a20 con 0x7fb450158db0 2023-06-23T18:36:04.402 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:04.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.399+0000 7fb454c98700 1 -- 172.21.15.79:0/2123616510 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb43006ace0 msgr2=0x7fb43006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.399+0000 7fb454c98700 1 --2- 172.21.15.79:0/2123616510 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb43006ace0 0x7fb43006d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb44400ace0 tx=0x7fb44400a320 comp rx=0 tx=0).stop 2023-06-23T18:36:04.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.399+0000 7fb454c98700 1 -- 172.21.15.79:0/2123616510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb450158db0 msgr2=0x7fb4501f2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.399+0000 7fb454c98700 1 --2- 172.21.15.79:0/2123616510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb450158db0 0x7fb4501f2c60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb440009c40 tx=0x7fb440004750 comp rx=0 tx=0).stop 2023-06-23T18:36:04.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.399+0000 7fb454c98700 1 -- 172.21.15.79:0/2123616510 shutdown_connections 2023-06-23T18:36:04.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.399+0000 7fb454c98700 1 --2- 172.21.15.79:0/2123616510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb45015be70 0x7fb45006ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.399+0000 7fb454c98700 1 --2- 172.21.15.79:0/2123616510 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb43006ace0 0x7fb43006d190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.399+0000 7fb454c98700 1 --2- 172.21.15.79:0/2123616510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4500cd040 0x7fb4500ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.399+0000 7fb454c98700 1 --2- 172.21.15.79:0/2123616510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb450158db0 0x7fb4501f2c60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.399+0000 7fb454c98700 1 -- 172.21.15.79:0/2123616510 >> 172.21.15.79:0/2123616510 conn(0x7fb4500cfc60 msgr2=0x7fb4501583f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:04.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.399+0000 7fb454c98700 1 -- 172.21.15.79:0/2123616510 shutdown_connections 2023-06-23T18:36:04.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.399+0000 7fb454c98700 1 -- 172.21.15.79:0/2123616510 wait complete. 2023-06-23T18:36:04.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:04.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.533+0000 7fc6c4118700 1 Processor -- start 2023-06-23T18:36:04.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.533+0000 7fc6c4118700 1 -- start start 2023-06-23T18:36:04.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.534+0000 7fc6c4118700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc6bc06a5a0 0x7fc6bc1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.534+0000 7fc6c4118700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6bc0cdc90 con 0x7fc6bc1569c0 2023-06-23T18:36:04.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.534+0000 7fc6c4118700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6bc15d0f0 con 0x7fc6bc0cd440 2023-06-23T18:36:04.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.534+0000 7fc6c4118700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6bc0cf570 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.534+0000 7fc6c16b3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc6bc06a5a0 0x7fc6bc1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.534+0000 7fc6c1eb4700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fc6bc1569c0 0x7fc6bc069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42708/0 (socket says 172.21.15.79:42708) 2023-06-23T18:36:04.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.534+0000 7fc6c1eb4700 1 -- 172.21.15.79:0/3302072931 learned_addr learned my addr 172.21.15.79:0/3302072931 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:04.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.534+0000 7fc6c16b3700 1 -- 172.21.15.79:0/3302072931 >> v1:172.21.15.155:6789/0 conn(0x7fc6bc0cd440 legacy=0x7fc6bc0b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:04.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.534+0000 7fc6c16b3700 1 -- 172.21.15.79:0/3302072931 >> v1:172.21.15.79:6789/0 conn(0x7fc6bc1569c0 legacy=0x7fc6bc069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.535+0000 7fc6c16b3700 1 -- 172.21.15.79:0/3302072931 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc156d90 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.535+0000 7fc6c0eb2700 1 -- 172.21.15.79:0/3302072931 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3796256429 0 0) 0x7fc6ac001110 con 0x7fc6bc1569c0 2023-06-23T18:36:04.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.535+0000 7fc6c16b3700 1 --2- 172.21.15.79:0/3302072931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc06a5a0 0x7fc6bc1608b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009840 tx=0x7fc6b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ef588da781b12e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:04.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.535+0000 7fc6c0eb2700 1 -- 172.21.15.79:0/3302072931 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8004230 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.535+0000 7fc6c0eb2700 1 -- 172.21.15.79:0/3302072931 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b8004390 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.535+0000 7fc6c0eb2700 1 -- 172.21.15.79:0/3302072931 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b80036a0 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.535+0000 7fc6c4118700 1 -- 172.21.15.79:0/3302072931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc06a5a0 msgr2=0x7fc6bc1608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.535+0000 7fc6c4118700 1 --2- 172.21.15.79:0/3302072931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc06a5a0 0x7fc6bc1608b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009840 tx=0x7fc6b8009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:04.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.535+0000 7fc6c4118700 1 -- 172.21.15.79:0/3302072931 shutdown_connections 2023-06-23T18:36:04.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.535+0000 7fc6c4118700 1 --2- 172.21.15.79:0/3302072931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc06a5a0 0x7fc6bc1608b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.544 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.535+0000 7fc6c4118700 1 -- 172.21.15.79:0/3302072931 >> 172.21.15.79:0/3302072931 conn(0x7fc6bc06afa0 msgr2=0x7fc6bc06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:04.544 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.535+0000 7fc6c4118700 1 -- 172.21.15.79:0/3302072931 shutdown_connections 2023-06-23T18:36:04.544 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.535+0000 7fc6c4118700 1 -- 172.21.15.79:0/3302072931 wait complete. 2023-06-23T18:36:04.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.537+0000 7fc6c4118700 1 Processor -- start 2023-06-23T18:36:04.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.537+0000 7fc6c4118700 1 -- start start 2023-06-23T18:36:04.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.537+0000 7fc6c4118700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc06a5a0 0x7fc6bc14ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.537+0000 7fc6c4118700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6bc0cd440 0x7fc6bc0b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.537+0000 7fc6c4118700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6bc1569c0 0x7fc6bc0c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.537+0000 7fc6c4118700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc0aba50 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.537+0000 7fc6c4118700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc0abd30 con 0x7fc6bc1569c0 2023-06-23T18:36:04.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.537+0000 7fc6c4118700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc0ac010 con 0x7fc6bc0cd440 2023-06-23T18:36:04.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.537+0000 7fc6c1eb4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc06a5a0 0x7fc6bc14ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.537+0000 7fc6c1eb4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc06a5a0 0x7fc6bc14ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39672/0 (socket says 172.21.15.79:39672) 2023-06-23T18:36:04.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.537+0000 7fc6c1eb4700 1 -- 172.21.15.79:0/3496305634 learned_addr learned my addr 172.21.15.79:0/3496305634 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:04.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6c26b5700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6bc1569c0 0x7fc6bc0c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6c16b3700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6bc0cd440 0x7fc6bc0b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6c1eb4700 1 -- 172.21.15.79:0/3496305634 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6bc0cd440 msgr2=0x7fc6bc0b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6c1eb4700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6bc0cd440 0x7fc6bc0b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6c1eb4700 1 -- 172.21.15.79:0/3496305634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6bc1569c0 msgr2=0x7fc6bc0c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6c1eb4700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6bc1569c0 0x7fc6bc0c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6c1eb4700 1 -- 172.21.15.79:0/3496305634 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b80094a0 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6c26b5700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6bc1569c0 0x7fc6bc0c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:04.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6c16b3700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6bc0cd440 0x7fc6bc0b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:04.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6c1eb4700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc06a5a0 0x7fc6bc14ffa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00c550 tx=0x7fc6ac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:04.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6b27fc700 1 -- 172.21.15.79:0/3496305634 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac004c80 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6b27fc700 1 -- 172.21.15.79:0/3496305634 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac004e00 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6b27fc700 1 -- 172.21.15.79:0/3496305634 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac007310 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6c4118700 1 -- 172.21.15.79:0/3496305634 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6bc0ac2f0 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.538+0000 7fc6c4118700 1 -- 172.21.15.79:0/3496305634 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc0b3a40 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.539+0000 7fc6c4118700 1 -- 172.21.15.79:0/3496305634 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6bc0672c0 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.539+0000 7fc6b27fc700 1 -- 172.21.15.79:0/3496305634 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc6ac007470 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.540+0000 7fc6b27fc700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc6a006ace0 0x7fc6a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.540+0000 7fc6c16b3700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc6a006ace0 0x7fc6a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.540+0000 7fc6b27fc700 1 -- 172.21.15.79:0/3496305634 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6ac089090 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.540+0000 7fc6c16b3700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc6a006ace0 0x7fc6a006d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc06a990 tx=0x7fc6b801c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:04.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.544+0000 7fc6b27fc700 1 -- 172.21.15.79:0/3496305634 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc6ac0540c0 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.733+0000 7fc6c4118700 1 -- 172.21.15.79:0/3496305634 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6bc0cdc90 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.734+0000 7fc6b27fc700 1 -- 172.21.15.79:0/3496305634 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6bc0cdc90 con 0x7fc6bc06a5a0 2023-06-23T18:36:04.739 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:04.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.736+0000 7fc6c4118700 1 -- 172.21.15.79:0/3496305634 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc6a006ace0 msgr2=0x7fc6a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.736+0000 7fc6c4118700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc6a006ace0 0x7fc6a006d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc06a990 tx=0x7fc6b801c040 comp rx=0 tx=0).stop 2023-06-23T18:36:04.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.736+0000 7fc6c4118700 1 -- 172.21.15.79:0/3496305634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc06a5a0 msgr2=0x7fc6bc14ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.736+0000 7fc6c4118700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc06a5a0 0x7fc6bc14ffa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00c550 tx=0x7fc6ac00c910 comp rx=0 tx=0).stop 2023-06-23T18:36:04.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.736+0000 7fc6c4118700 1 -- 172.21.15.79:0/3496305634 shutdown_connections 2023-06-23T18:36:04.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.736+0000 7fc6c4118700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc6a006ace0 0x7fc6a006d190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.736+0000 7fc6c4118700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6bc1569c0 0x7fc6bc0c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.736+0000 7fc6c4118700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc06a5a0 0x7fc6bc14ffa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.736+0000 7fc6c4118700 1 --2- 172.21.15.79:0/3496305634 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6bc0cd440 0x7fc6bc0b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.736+0000 7fc6c4118700 1 -- 172.21.15.79:0/3496305634 >> 172.21.15.79:0/3496305634 conn(0x7fc6bc06afa0 msgr2=0x7fc6bc0cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:04.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.736+0000 7fc6c4118700 1 -- 172.21.15.79:0/3496305634 shutdown_connections 2023-06-23T18:36:04.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.736+0000 7fc6c4118700 1 -- 172.21.15.79:0/3496305634 wait complete. 2023-06-23T18:36:04.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:36:04.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.871+0000 7f399b3f0700 1 Processor -- start 2023-06-23T18:36:04.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.871+0000 7f399b3f0700 1 -- start start 2023-06-23T18:36:04.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f399b3f0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399415a380 0x7f39940ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f399b3f0700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f39940cea30 0x7f39940b1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f399b3f0700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39940cee70 con 0x7f399415a750 2023-06-23T18:36:04.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f399b3f0700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39940b1d40 con 0x7f39940cea30 2023-06-23T18:36:04.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f399b3f0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39940b2020 con 0x7f399415a380 2023-06-23T18:36:04.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f399898b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f399415a750 0x7f39940ade50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42722/0 (socket says 172.21.15.79:42722) 2023-06-23T18:36:04.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f399898b700 1 -- 172.21.15.79:0/1990483239 learned_addr learned my addr 172.21.15.79:0/1990483239 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:04.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 -- 172.21.15.79:0/1990483239 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041712726 0 0) 0x7f39940cee70 con 0x7f399415a750 2023-06-23T18:36:04.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f399998d700 1 --2- 172.21.15.79:0/1990483239 >> v2:172.21.15.155:3300/0 conn(0x7f39940cea30 0x7f39940b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f399918c700 1 --2- 172.21.15.79:0/1990483239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399415a380 0x7f39940ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 -- 172.21.15.79:0/1990483239 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39800035b0 con 0x7f399415a750 2023-06-23T18:36:04.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 -- 172.21.15.79:0/1990483239 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 276437732 0 0) 0x7f39800035b0 con 0x7f399415a750 2023-06-23T18:36:04.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 -- 172.21.15.79:0/1990483239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399415a380 msgr2=0x7f39940ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:04.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 --2- 172.21.15.79:0/1990483239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399415a380 0x7f39940ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 -- 172.21.15.79:0/1990483239 >> v2:172.21.15.155:3300/0 conn(0x7f39940cea30 msgr2=0x7f39940b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:04.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 --2- 172.21.15.79:0/1990483239 >> v2:172.21.15.155:3300/0 conn(0x7f39940cea30 0x7f39940b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 -- 172.21.15.79:0/1990483239 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39940ad650 con 0x7f399415a750 2023-06-23T18:36:04.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 -- 172.21.15.79:0/1990483239 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f3990001670 con 0x7f399415a750 2023-06-23T18:36:04.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 -- 172.21.15.79:0/1990483239 >> v1:172.21.15.79:6789/0 conn(0x7f399415a750 legacy=0x7f39940ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 --2- 172.21.15.79:0/1990483239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399415a380 0x7f3980003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 --2- 172.21.15.79:0/1990483239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3980003f80 0x7f3980004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 --2- 172.21.15.79:0/1990483239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3980004d90 0x7f3980005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 -- 172.21.15.79:0/1990483239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940b1d40 con 0x7f3980004d90 2023-06-23T18:36:04.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 -- 172.21.15.79:0/1990483239 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940b2020 con 0x7f3980003f80 2023-06-23T18:36:04.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.872+0000 7f398bfff700 1 -- 172.21.15.79:0/1990483239 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398000ba10 con 0x7f399415a380 2023-06-23T18:36:04.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.873+0000 7f399998d700 1 --2- 172.21.15.79:0/1990483239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3980004d90 0x7f3980005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.873+0000 7f399918c700 1 --2- 172.21.15.79:0/1990483239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399415a380 0x7f3980003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.873+0000 7f399898b700 1 --2- 172.21.15.79:0/1990483239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3980003f80 0x7f3980004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.873+0000 7f399998d700 1 -- 172.21.15.79:0/1990483239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399415a380 msgr2=0x7f3980003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:04.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.873+0000 7f399998d700 1 --2- 172.21.15.79:0/1990483239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399415a380 0x7f3980003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.873+0000 7f399998d700 1 -- 172.21.15.79:0/1990483239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3980003f80 msgr2=0x7f3980004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:04.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.873+0000 7f399998d700 1 --2- 172.21.15.79:0/1990483239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3980003f80 0x7f3980004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.873+0000 7f399998d700 1 -- 172.21.15.79:0/1990483239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39800035b0 con 0x7f3980004d90 2023-06-23T18:36:04.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.873+0000 7f399998d700 1 --2- 172.21.15.79:0/1990483239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3980004d90 0x7f3980005200 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f398c009fc0 tx=0x7f3980005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1d84400536609c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:04.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.873+0000 7f398bfff700 1 -- 172.21.15.79:0/1990483239 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c00ec30 con 0x7f3980004d90 2023-06-23T18:36:04.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.873+0000 7f398bfff700 1 -- 172.21.15.79:0/1990483239 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c00f040 con 0x7f3980004d90 2023-06-23T18:36:04.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.873+0000 7f398bfff700 1 -- 172.21.15.79:0/1990483239 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c014480 con 0x7f3980004d90 2023-06-23T18:36:04.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.874+0000 7f399b3f0700 1 -- 172.21.15.79:0/1990483239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3980004d90 msgr2=0x7f3980005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.874+0000 7f399b3f0700 1 --2- 172.21.15.79:0/1990483239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3980004d90 0x7f3980005200 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f398c009fc0 tx=0x7f3980005880 comp rx=0 tx=0).stop 2023-06-23T18:36:04.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.874+0000 7f399998d700 1 -- 172.21.15.79:0/1990483239 reap_dead start 2023-06-23T18:36:04.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.874+0000 7f399b3f0700 1 -- 172.21.15.79:0/1990483239 shutdown_connections 2023-06-23T18:36:04.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.874+0000 7f399b3f0700 1 -- 172.21.15.79:0/1990483239 >> 172.21.15.79:0/1990483239 conn(0x7f3994089fc0 msgr2=0x7f3994099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:04.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.874+0000 7f399b3f0700 1 -- 172.21.15.79:0/1990483239 shutdown_connections 2023-06-23T18:36:04.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.874+0000 7f399b3f0700 1 -- 172.21.15.79:0/1990483239 wait complete. 2023-06-23T18:36:04.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.875+0000 7f399b3f0700 1 Processor -- start 2023-06-23T18:36:04.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.875+0000 7f399b3f0700 1 -- start start 2023-06-23T18:36:04.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.875+0000 7f399b3f0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f399414c700 0x7f39941f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.875+0000 7f399b3f0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f399414cad0 0x7f39941f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.875+0000 7f399b3f0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399415d550 0x7f39941f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399b3f0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940c4e30 con 0x7f399414cad0 2023-06-23T18:36:04.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399b3f0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940c5b50 con 0x7f399414c700 2023-06-23T18:36:04.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399b3f0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39940c5080 con 0x7f399415d550 2023-06-23T18:36:04.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399898b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f399414cad0 0x7f39941f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399898b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f399414cad0 0x7f39941f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39694/0 (socket says 172.21.15.79:39694) 2023-06-23T18:36:04.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399898b700 1 -- 172.21.15.79:0/626468439 learned_addr learned my addr 172.21.15.79:0/626468439 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:04.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399998d700 1 --2- 172.21.15.79:0/626468439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399415d550 0x7f39941f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399918c700 1 --2- 172.21.15.79:0/626468439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f399414c700 0x7f39941f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399898b700 1 -- 172.21.15.79:0/626468439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399415d550 msgr2=0x7f39941f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399898b700 1 --2- 172.21.15.79:0/626468439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399415d550 0x7f39941f41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399898b700 1 -- 172.21.15.79:0/626468439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f399414c700 msgr2=0x7f39941f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:04.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399898b700 1 --2- 172.21.15.79:0/626468439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f399414c700 0x7f39941f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:04.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399898b700 1 -- 172.21.15.79:0/626468439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39940c5360 con 0x7f399414cad0 2023-06-23T18:36:04.897 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399898b700 1 --2- 172.21.15.79:0/626468439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f399414cad0 0x7f39941f3c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3990007310 tx=0x7f39900076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:04.897 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f3989ffb700 1 -- 172.21.15.79:0/626468439 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399000f8d0 con 0x7f399414cad0 2023-06-23T18:36:04.897 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f3989ffb700 1 -- 172.21.15.79:0/626468439 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399000fa50 con 0x7f399414cad0 2023-06-23T18:36:04.898 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399b3f0700 1 -- 172.21.15.79:0/626468439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39940c5640 con 0x7f399414cad0 2023-06-23T18:36:04.898 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f3989ffb700 1 -- 172.21.15.79:0/626468439 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399000fe00 con 0x7f399414cad0 2023-06-23T18:36:04.898 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.876+0000 7f399b3f0700 1 -- 172.21.15.79:0/626468439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39940b4a70 con 0x7f399414cad0 2023-06-23T18:36:04.898 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.877+0000 7f399b3f0700 1 -- 172.21.15.79:0/626468439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39940b3360 con 0x7f399414cad0 2023-06-23T18:36:04.900 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.878+0000 7f3989ffb700 1 -- 172.21.15.79:0/626468439 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f399001d460 con 0x7f399414cad0 2023-06-23T18:36:04.900 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.878+0000 7f3989ffb700 1 --2- 172.21.15.79:0/626468439 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f397806ace0 0x7f397806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:04.900 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.878+0000 7f399918c700 1 --2- 172.21.15.79:0/626468439 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f397806ace0 0x7f397806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:04.901 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.878+0000 7f3989ffb700 1 -- 172.21.15.79:0/626468439 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39900895e0 con 0x7f399414cad0 2023-06-23T18:36:04.901 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.878+0000 7f399918c700 1 --2- 172.21.15.79:0/626468439 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f397806ace0 0x7f397806d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3984009b60 tx=0x7f3984008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:04.901 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:04.882+0000 7f3989ffb700 1 -- 172.21.15.79:0/626468439 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3990054720 con 0x7f399414cad0 2023-06-23T18:36:05.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.111+0000 7f399b3f0700 1 -- 172.21.15.79:0/626468439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39940c5d10 con 0x7f399414cad0 2023-06-23T18:36:05.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.112+0000 7f3989ffb700 1 -- 172.21.15.79:0/626468439 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f3990057d40 con 0x7f399414cad0 2023-06-23T18:36:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/399719238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":399719238},{"type":"v1","addr":"172.21.15.155:6834","nonce":399719238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1846440916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1846440916},{"type":"v1","addr":"172.21.15.155:6835","nonce":1846440916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3310180303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3310180303},{"type":"v1","addr":"172.21.15.79:6835","nonce":3310180303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:35:53.511495+0000","modified":"2023-06-23T18:35:56.527609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6837/3900137670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3900137670},{"type":"v1","addr":"172.21.15.79:6837","nonce":3900137670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:36:05.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.114+0000 7f399b3f0700 1 -- 172.21.15.79:0/626468439 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f397806ace0 msgr2=0x7f397806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.114+0000 7f399b3f0700 1 --2- 172.21.15.79:0/626468439 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f397806ace0 0x7f397806d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3984009b60 tx=0x7f3984008040 comp rx=0 tx=0).stop 2023-06-23T18:36:05.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.114+0000 7f399b3f0700 1 -- 172.21.15.79:0/626468439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f399414cad0 msgr2=0x7f39941f3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.114+0000 7f399b3f0700 1 --2- 172.21.15.79:0/626468439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f399414cad0 0x7f39941f3c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3990007310 tx=0x7f39900076d0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.114+0000 7f399b3f0700 1 -- 172.21.15.79:0/626468439 shutdown_connections 2023-06-23T18:36:05.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.114+0000 7f399b3f0700 1 --2- 172.21.15.79:0/626468439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f399415d550 0x7f39941f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.114+0000 7f399b3f0700 1 --2- 172.21.15.79:0/626468439 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f397806ace0 0x7f397806d190 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.114+0000 7f399b3f0700 1 --2- 172.21.15.79:0/626468439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f399414c700 0x7f39941f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.114+0000 7f399b3f0700 1 --2- 172.21.15.79:0/626468439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f399414cad0 0x7f39941f3c80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.114+0000 7f399b3f0700 1 -- 172.21.15.79:0/626468439 >> 172.21.15.79:0/626468439 conn(0x7f3994089fc0 msgr2=0x7f39940b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:05.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.114+0000 7f399b3f0700 1 -- 172.21.15.79:0/626468439 shutdown_connections 2023-06-23T18:36:05.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.114+0000 7f399b3f0700 1 -- 172.21.15.79:0/626468439 wait complete. 2023-06-23T18:36:05.121 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 8 2023-06-23T18:36:05.132 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:05.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.247+0000 7f581b1d9700 1 Processor -- start 2023-06-23T18:36:05.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.247+0000 7f581b1d9700 1 -- start start 2023-06-23T18:36:05.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.248+0000 7f581b1d9700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f58140aee30 0x7f581415d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.248+0000 7f581b1d9700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58141608f0 con 0x7f5814071490 2023-06-23T18:36:05.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.248+0000 7f581b1d9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5814160e20 con 0x7f58140b1270 2023-06-23T18:36:05.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.248+0000 7f581b1d9700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58140cd4f0 con 0x7f58140aee30 2023-06-23T18:36:05.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.248+0000 7f5819776700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f58140aee30 0x7f581415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.248+0000 7f5818f75700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5814071490 0x7f5814071860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42742/0 (socket says 172.21.15.79:42742) 2023-06-23T18:36:05.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.248+0000 7f5818f75700 1 -- 172.21.15.79:0/1963515888 learned_addr learned my addr 172.21.15.79:0/1963515888 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:05.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f5819776700 1 -- 172.21.15.79:0/1963515888 >> v1:172.21.15.155:6789/0 conn(0x7f58140b1270 legacy=0x7f581415c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:05.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f5819776700 1 -- 172.21.15.79:0/1963515888 >> v1:172.21.15.79:6789/0 conn(0x7f5814071490 legacy=0x7f5814071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f5819776700 1 -- 172.21.15.79:0/1963515888 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5814160b40 con 0x7f58140aee30 2023-06-23T18:36:05.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f58137fe700 1 -- 172.21.15.79:0/1963515888 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367877419 0 0) 0x7f58141608f0 con 0x7f5814071490 2023-06-23T18:36:05.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f5819776700 1 --2- 172.21.15.79:0/1963515888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58140aee30 0x7f581415d100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f580c009d90 tx=0x7f580c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40643abd6708fa47 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:05.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f58137fe700 1 -- 172.21.15.79:0/1963515888 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c00e070 con 0x7f58140aee30 2023-06-23T18:36:05.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f58137fe700 1 -- 172.21.15.79:0/1963515888 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c00bb90 con 0x7f58140aee30 2023-06-23T18:36:05.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f58137fe700 1 -- 172.21.15.79:0/1963515888 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c003a20 con 0x7f58140aee30 2023-06-23T18:36:05.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f581b1d9700 1 -- 172.21.15.79:0/1963515888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58140aee30 msgr2=0x7f581415d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f581b1d9700 1 --2- 172.21.15.79:0/1963515888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58140aee30 0x7f581415d100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f580c009d90 tx=0x7f580c00b330 comp rx=0 tx=0).stop 2023-06-23T18:36:05.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f581b1d9700 1 -- 172.21.15.79:0/1963515888 shutdown_connections 2023-06-23T18:36:05.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f581b1d9700 1 --2- 172.21.15.79:0/1963515888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58140aee30 0x7f581415d100 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f581b1d9700 1 -- 172.21.15.79:0/1963515888 >> 172.21.15.79:0/1963515888 conn(0x7f581406d660 msgr2=0x7f581406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:05.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f581b1d9700 1 -- 172.21.15.79:0/1963515888 shutdown_connections 2023-06-23T18:36:05.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.249+0000 7f581b1d9700 1 -- 172.21.15.79:0/1963515888 wait complete. 2023-06-23T18:36:05.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.251+0000 7f581b1d9700 1 Processor -- start 2023-06-23T18:36:05.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.251+0000 7f581b1d9700 1 -- start start 2023-06-23T18:36:05.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.251+0000 7f581b1d9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5814071490 0x7f581414d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.251+0000 7f581b1d9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58140aee30 0x7f58140c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.251+0000 7f581b1d9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58140b1270 0x7f581414cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.251+0000 7f581b1d9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58140b2a40 con 0x7f58140aee30 2023-06-23T18:36:05.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.251+0000 7f581b1d9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581414d070 con 0x7f5814071490 2023-06-23T18:36:05.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.251+0000 7f581b1d9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58140b3e50 con 0x7f58140b1270 2023-06-23T18:36:05.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f5813fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58140aee30 0x7f58140c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f5813fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58140aee30 0x7f58140c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39708/0 (socket says 172.21.15.79:39708) 2023-06-23T18:36:05.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f5813fff700 1 -- 172.21.15.79:0/1307457348 learned_addr learned my addr 172.21.15.79:0/1307457348 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:05.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f5818f75700 1 --2- 172.21.15.79:0/1307457348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5814071490 0x7f581414d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f5819776700 1 --2- 172.21.15.79:0/1307457348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58140b1270 0x7f581414cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f5813fff700 1 -- 172.21.15.79:0/1307457348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58140b1270 msgr2=0x7f581414cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f5813fff700 1 --2- 172.21.15.79:0/1307457348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58140b1270 0x7f581414cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f5813fff700 1 -- 172.21.15.79:0/1307457348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5814071490 msgr2=0x7f581414d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f5813fff700 1 --2- 172.21.15.79:0/1307457348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5814071490 0x7f581414d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f5813fff700 1 -- 172.21.15.79:0/1307457348 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c009a70 con 0x7f58140aee30 2023-06-23T18:36:05.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f5813fff700 1 --2- 172.21.15.79:0/1307457348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58140aee30 0x7f58140c5de0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5800006fd0 tx=0x7f580000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:05.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f58117fa700 1 -- 172.21.15.79:0/1307457348 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800004020 con 0x7f58140aee30 2023-06-23T18:36:05.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f58117fa700 1 -- 172.21.15.79:0/1307457348 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5800004180 con 0x7f58140aee30 2023-06-23T18:36:05.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f58117fa700 1 -- 172.21.15.79:0/1307457348 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800004510 con 0x7f58140aee30 2023-06-23T18:36:05.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f581b1d9700 1 -- 172.21.15.79:0/1307457348 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58140b40d0 con 0x7f58140aee30 2023-06-23T18:36:05.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.252+0000 7f581b1d9700 1 -- 172.21.15.79:0/1307457348 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58140b4690 con 0x7f58140aee30 2023-06-23T18:36:05.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.253+0000 7f581b1d9700 1 -- 172.21.15.79:0/1307457348 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58140b0270 con 0x7f58140aee30 2023-06-23T18:36:05.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.254+0000 7f58117fa700 1 -- 172.21.15.79:0/1307457348 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5800003680 con 0x7f58140aee30 2023-06-23T18:36:05.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.254+0000 7f58117fa700 1 --2- 172.21.15.79:0/1307457348 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f57f806ad10 0x7f57f806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.254+0000 7f58117fa700 1 -- 172.21.15.79:0/1307457348 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5800088f80 con 0x7f58140aee30 2023-06-23T18:36:05.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.254+0000 7f5818f75700 1 --2- 172.21.15.79:0/1307457348 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f57f806ad10 0x7f57f806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.254+0000 7f5818f75700 1 --2- 172.21.15.79:0/1307457348 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f57f806ad10 0x7f57f806d1c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5808000f60 tx=0x7f5808007c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:05.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.258+0000 7f58117fa700 1 -- 172.21.15.79:0/1307457348 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f58000bb310 con 0x7f58140aee30 2023-06-23T18:36:05.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.448+0000 7f581b1d9700 1 -- 172.21.15.79:0/1307457348 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58140724a0 con 0x7f58140aee30 2023-06-23T18:36:05.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.450+0000 7f58117fa700 1 -- 172.21.15.79:0/1307457348 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7f58000540c0 con 0x7f58140aee30 2023-06-23T18:36:05.455 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:05.455 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:05.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.452+0000 7f581b1d9700 1 -- 172.21.15.79:0/1307457348 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f57f806ad10 msgr2=0x7f57f806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.452+0000 7f581b1d9700 1 --2- 172.21.15.79:0/1307457348 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f57f806ad10 0x7f57f806d1c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5808000f60 tx=0x7f5808007c90 comp rx=0 tx=0).stop 2023-06-23T18:36:05.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.452+0000 7f581b1d9700 1 -- 172.21.15.79:0/1307457348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58140aee30 msgr2=0x7f58140c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.452+0000 7f581b1d9700 1 --2- 172.21.15.79:0/1307457348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58140aee30 0x7f58140c5de0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5800006fd0 tx=0x7f580000b7e0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.452+0000 7f581b1d9700 1 -- 172.21.15.79:0/1307457348 shutdown_connections 2023-06-23T18:36:05.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.452+0000 7f581b1d9700 1 --2- 172.21.15.79:0/1307457348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58140b1270 0x7f581414cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.452+0000 7f581b1d9700 1 --2- 172.21.15.79:0/1307457348 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f57f806ad10 0x7f57f806d1c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.452+0000 7f581b1d9700 1 --2- 172.21.15.79:0/1307457348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5814071490 0x7f581414d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.452+0000 7f581b1d9700 1 --2- 172.21.15.79:0/1307457348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58140aee30 0x7f58140c5de0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.452+0000 7f581b1d9700 1 -- 172.21.15.79:0/1307457348 >> 172.21.15.79:0/1307457348 conn(0x7f581406d660 msgr2=0x7f58140b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:05.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.452+0000 7f581b1d9700 1 -- 172.21.15.79:0/1307457348 shutdown_connections 2023-06-23T18:36:05.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.452+0000 7f581b1d9700 1 -- 172.21.15.79:0/1307457348 wait complete. 2023-06-23T18:36:05.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:05.471 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:05.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.588+0000 7f92c838c700 1 Processor -- start 2023-06-23T18:36:05.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.588+0000 7f92c838c700 1 -- start start 2023-06-23T18:36:05.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.589+0000 7f92c838c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92c01569c0 0x7f92c01608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.589+0000 7f92c838c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f92c006a5a0 0x7f92c00b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.589+0000 7f92c838c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c00cdc90 con 0x7f92c00cd440 2023-06-23T18:36:05.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.589+0000 7f92c838c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c00cf570 con 0x7f92c006a5a0 2023-06-23T18:36:05.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.589+0000 7f92c838c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c015c8b0 con 0x7f92c01569c0 2023-06-23T18:36:05.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.589+0000 7f92c6929700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f92c006a5a0 0x7f92c00b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.589+0000 7f92c6929700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f92c006a5a0 0x7f92c00b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39718/0 (socket says 172.21.15.79:39718) 2023-06-23T18:36:05.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.589+0000 7f92c6929700 1 -- 172.21.15.79:0/81903866 learned_addr learned my addr 172.21.15.79:0/81903866 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:05.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.590+0000 7f92c6128700 1 --2- 172.21.15.79:0/81903866 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92c01569c0 0x7f92c01608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.590+0000 7f92c6929700 1 -- 172.21.15.79:0/81903866 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92c01569c0 msgr2=0x7f92c01608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:05.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.590+0000 7f92c6929700 1 --2- 172.21.15.79:0/81903866 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92c01569c0 0x7f92c01608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.590+0000 7f92c6929700 1 -- 172.21.15.79:0/81903866 >> v1:172.21.15.155:6789/0 conn(0x7f92c00cd440 legacy=0x7f92c0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.590+0000 7f92c6929700 1 -- 172.21.15.79:0/81903866 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c015d0f0 con 0x7f92c006a5a0 2023-06-23T18:36:05.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.590+0000 7f92c6929700 1 --2- 172.21.15.79:0/81903866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92c006a5a0 0x7f92c00b12f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009e50 tx=0x7f92b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e72d5f450aeb988 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:05.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.590+0000 7f92c5126700 1 -- 172.21.15.79:0/81903866 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b800e070 con 0x7f92c006a5a0 2023-06-23T18:36:05.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.590+0000 7f92c5126700 1 -- 172.21.15.79:0/81903866 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b800b9c0 con 0x7f92c006a5a0 2023-06-23T18:36:05.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.590+0000 7f92c5126700 1 -- 172.21.15.79:0/81903866 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b800bd70 con 0x7f92c006a5a0 2023-06-23T18:36:05.599 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.590+0000 7f92c838c700 1 -- 172.21.15.79:0/81903866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92c006a5a0 msgr2=0x7f92c00b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.599 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.590+0000 7f92c838c700 1 --2- 172.21.15.79:0/81903866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92c006a5a0 0x7f92c00b12f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009e50 tx=0x7f92b800b330 comp rx=0 tx=0).stop 2023-06-23T18:36:05.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.591+0000 7f92c838c700 1 -- 172.21.15.79:0/81903866 shutdown_connections 2023-06-23T18:36:05.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.591+0000 7f92c838c700 1 --2- 172.21.15.79:0/81903866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92c006a5a0 0x7f92c00b12f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.591+0000 7f92c838c700 1 --2- 172.21.15.79:0/81903866 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92c01569c0 0x7f92c01608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.591+0000 7f92c838c700 1 -- 172.21.15.79:0/81903866 >> 172.21.15.79:0/81903866 conn(0x7f92c006afa0 msgr2=0x7f92c006b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:05.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.591+0000 7f92c838c700 1 -- 172.21.15.79:0/81903866 shutdown_connections 2023-06-23T18:36:05.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.591+0000 7f92c838c700 1 -- 172.21.15.79:0/81903866 wait complete. 2023-06-23T18:36:05.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.592+0000 7f92c838c700 1 Processor -- start 2023-06-23T18:36:05.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.592+0000 7f92c838c700 1 -- start start 2023-06-23T18:36:05.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.592+0000 7f92c838c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92c006a5a0 0x7f92c00c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.592+0000 7f92c838c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92c00cd440 0x7f92c01f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.592+0000 7f92c838c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92c01569c0 0x7f92c01f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.592+0000 7f92c838c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00c9860 con 0x7f92c01569c0 2023-06-23T18:36:05.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.592+0000 7f92c838c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00c3ee0 con 0x7f92c006a5a0 2023-06-23T18:36:05.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.592+0000 7f92c838c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00c41c0 con 0x7f92c00cd440 2023-06-23T18:36:05.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92c6929700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92c01569c0 0x7f92c01f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92c6929700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92c01569c0 0x7f92c01f5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39724/0 (socket says 172.21.15.79:39724) 2023-06-23T18:36:05.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92c6929700 1 -- 172.21.15.79:0/3225296264 learned_addr learned my addr 172.21.15.79:0/3225296264 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:05.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92c6128700 1 --2- 172.21.15.79:0/3225296264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92c006a5a0 0x7f92c00c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92c5927700 1 --2- 172.21.15.79:0/3225296264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92c00cd440 0x7f92c01f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92c6929700 1 -- 172.21.15.79:0/3225296264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92c00cd440 msgr2=0x7f92c01f13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92c6929700 1 --2- 172.21.15.79:0/3225296264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92c00cd440 0x7f92c01f13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92c6929700 1 -- 172.21.15.79:0/3225296264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92c006a5a0 msgr2=0x7f92c00c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92c6929700 1 --2- 172.21.15.79:0/3225296264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92c006a5a0 0x7f92c00c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92c6929700 1 -- 172.21.15.79:0/3225296264 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b8009b30 con 0x7f92c01569c0 2023-06-23T18:36:05.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92c6929700 1 --2- 172.21.15.79:0/3225296264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92c01569c0 0x7f92c01f5090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f92b8012000 tx=0x7f92b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:05.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92b6ffd700 1 -- 172.21.15.79:0/3225296264 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8004800 con 0x7f92c01569c0 2023-06-23T18:36:05.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92b6ffd700 1 -- 172.21.15.79:0/3225296264 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b8004960 con 0x7f92c01569c0 2023-06-23T18:36:05.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92c838c700 1 -- 172.21.15.79:0/3225296264 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c00c44a0 con 0x7f92c01569c0 2023-06-23T18:36:05.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92b6ffd700 1 -- 172.21.15.79:0/3225296264 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b801c680 con 0x7f92c01569c0 2023-06-23T18:36:05.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.593+0000 7f92c838c700 1 -- 172.21.15.79:0/3225296264 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c01f1900 con 0x7f92c01569c0 2023-06-23T18:36:05.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.594+0000 7f92c838c700 1 -- 172.21.15.79:0/3225296264 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c00672c0 con 0x7f92c01569c0 2023-06-23T18:36:05.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.594+0000 7f92b6ffd700 1 -- 172.21.15.79:0/3225296264 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f92b801c7e0 con 0x7f92c01569c0 2023-06-23T18:36:05.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.595+0000 7f92b6ffd700 1 --2- 172.21.15.79:0/3225296264 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f92a406ace0 0x7f92a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.595+0000 7f92b6ffd700 1 -- 172.21.15.79:0/3225296264 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f92b808acc0 con 0x7f92c01569c0 2023-06-23T18:36:05.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.595+0000 7f92c6128700 1 --2- 172.21.15.79:0/3225296264 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f92a406ace0 0x7f92a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.595+0000 7f92c6128700 1 --2- 172.21.15.79:0/3225296264 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f92a406ace0 0x7f92a406d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f92c00c64a0 tx=0x7f92b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:05.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.599+0000 7f92b6ffd700 1 -- 172.21.15.79:0/3225296264 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f92b8055e00 con 0x7f92c01569c0 2023-06-23T18:36:05.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.786+0000 7f92c838c700 1 -- 172.21.15.79:0/3225296264 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f92c00c6700 con 0x7f92c01569c0 2023-06-23T18:36:05.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.786+0000 7f92b6ffd700 1 -- 172.21.15.79:0/3225296264 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f92b8059420 con 0x7f92c01569c0 2023-06-23T18:36:05.792 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:05.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.788+0000 7f92c838c700 1 -- 172.21.15.79:0/3225296264 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f92a406ace0 msgr2=0x7f92a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.788+0000 7f92c838c700 1 --2- 172.21.15.79:0/3225296264 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f92a406ace0 0x7f92a406d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f92c00c64a0 tx=0x7f92b0008040 comp rx=0 tx=0).stop 2023-06-23T18:36:05.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.788+0000 7f92c838c700 1 -- 172.21.15.79:0/3225296264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92c01569c0 msgr2=0x7f92c01f5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.788+0000 7f92c838c700 1 --2- 172.21.15.79:0/3225296264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92c01569c0 0x7f92c01f5090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f92b8012000 tx=0x7f92b8004ad0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.788+0000 7f92c838c700 1 -- 172.21.15.79:0/3225296264 shutdown_connections 2023-06-23T18:36:05.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.788+0000 7f92c838c700 1 --2- 172.21.15.79:0/3225296264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92c01569c0 0x7f92c01f5090 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.788+0000 7f92c838c700 1 --2- 172.21.15.79:0/3225296264 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f92a406ace0 0x7f92a406d190 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.788+0000 7f92c838c700 1 --2- 172.21.15.79:0/3225296264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92c006a5a0 0x7f92c00c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.788+0000 7f92c838c700 1 --2- 172.21.15.79:0/3225296264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92c00cd440 0x7f92c01f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.788+0000 7f92c838c700 1 -- 172.21.15.79:0/3225296264 >> 172.21.15.79:0/3225296264 conn(0x7f92c006afa0 msgr2=0x7f92c01607b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:05.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.788+0000 7f92c838c700 1 -- 172.21.15.79:0/3225296264 shutdown_connections 2023-06-23T18:36:05.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.788+0000 7f92c838c700 1 -- 172.21.15.79:0/3225296264 wait complete. 2023-06-23T18:36:05.805 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:05.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.922+0000 7efffbe3b700 1 Processor -- start 2023-06-23T18:36:05.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.922+0000 7efffbe3b700 1 -- start start 2023-06-23T18:36:05.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efffbe3b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efff415c710 0x7efff40b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efffbe3b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efff40cd410 0x7efff40b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efffbe3b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff415d620 con 0x7efff40aed90 2023-06-23T18:36:05.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efffbe3b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efff40ca870 con 0x7efff415c710 2023-06-23T18:36:05.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efffbe3b700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efff40724a0 con 0x7efff40cd410 2023-06-23T18:36:05.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efff9bd7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efff415c710 0x7efff40b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efff93d6700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7efff40aed90 0x7efff406eeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42770/0 (socket says 172.21.15.79:42770) 2023-06-23T18:36:05.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efff93d6700 1 -- 172.21.15.79:0/652415090 learned_addr learned my addr 172.21.15.79:0/652415090 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:05.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efffa3d8700 1 --2- 172.21.15.79:0/652415090 >> v2:172.21.15.155:3300/0 conn(0x7efff40cd410 0x7efff40b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efff9bd7700 1 -- 172.21.15.79:0/652415090 >> v2:172.21.15.155:3300/0 conn(0x7efff40cd410 msgr2=0x7efff40b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:05.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efff9bd7700 1 --2- 172.21.15.79:0/652415090 >> v2:172.21.15.155:3300/0 conn(0x7efff40cd410 0x7efff40b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efff9bd7700 1 -- 172.21.15.79:0/652415090 >> v1:172.21.15.79:6789/0 conn(0x7efff40aed90 legacy=0x7efff406eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efff9bd7700 1 -- 172.21.15.79:0/652415090 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff40cd9d0 con 0x7efff415c710 2023-06-23T18:36:05.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efff8bd5700 1 -- 172.21.15.79:0/652415090 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478128740 0 0) 0x7efff415d620 con 0x7efff40aed90 2023-06-23T18:36:05.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.923+0000 7efffa3d8700 1 --2- 172.21.15.79:0/652415090 >> v2:172.21.15.155:3300/0 conn(0x7efff40cd410 0x7efff40b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:05.931 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.924+0000 7efff9bd7700 1 --2- 172.21.15.79:0/652415090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efff415c710 0x7efff40b12c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efff40ce060 tx=0x7effe4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780478eb7b2d39ed server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:05.931 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.924+0000 7efff8bd5700 1 -- 172.21.15.79:0/652415090 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4011830 con 0x7efff415c710 2023-06-23T18:36:05.931 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.924+0000 7efff8bd5700 1 -- 172.21.15.79:0/652415090 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe4011990 con 0x7efff415c710 2023-06-23T18:36:05.931 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.924+0000 7efff8bd5700 1 -- 172.21.15.79:0/652415090 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4011d20 con 0x7efff415c710 2023-06-23T18:36:05.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.924+0000 7efffbe3b700 1 -- 172.21.15.79:0/652415090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efff415c710 msgr2=0x7efff40b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.924+0000 7efffbe3b700 1 --2- 172.21.15.79:0/652415090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efff415c710 0x7efff40b12c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efff40ce060 tx=0x7effe4009b20 comp rx=0 tx=0).stop 2023-06-23T18:36:05.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.924+0000 7efffbe3b700 1 -- 172.21.15.79:0/652415090 shutdown_connections 2023-06-23T18:36:05.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.924+0000 7efffbe3b700 1 --2- 172.21.15.79:0/652415090 >> v2:172.21.15.155:3300/0 conn(0x7efff40cd410 0x7efff40b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.924+0000 7efffbe3b700 1 --2- 172.21.15.79:0/652415090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efff415c710 0x7efff40b12c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.924+0000 7efffbe3b700 1 -- 172.21.15.79:0/652415090 >> 172.21.15.79:0/652415090 conn(0x7efff406e200 msgr2=0x7efff40cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:05.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.924+0000 7efffbe3b700 1 -- 172.21.15.79:0/652415090 shutdown_connections 2023-06-23T18:36:05.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.924+0000 7efffbe3b700 1 -- 172.21.15.79:0/652415090 wait complete. 2023-06-23T18:36:05.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.926+0000 7efffbe3b700 1 Processor -- start 2023-06-23T18:36:05.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.926+0000 7efffbe3b700 1 -- start start 2023-06-23T18:36:05.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.926+0000 7efffbe3b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efff40aed90 0x7efff40c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.926+0000 7efffbe3b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efff40cd410 0x7efff40b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.926+0000 7efffbe3b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efff415c710 0x7efff40b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.926+0000 7efffbe3b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff40b3c60 con 0x7efff415c710 2023-06-23T18:36:05.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.926+0000 7efffbe3b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff40b2fe0 con 0x7efff40cd410 2023-06-23T18:36:05.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.926+0000 7efffbe3b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff40b32c0 con 0x7efff40aed90 2023-06-23T18:36:05.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.926+0000 7efffa3d8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efff415c710 0x7efff40b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.926+0000 7efffa3d8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efff415c710 0x7efff40b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39736/0 (socket says 172.21.15.79:39736) 2023-06-23T18:36:05.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.926+0000 7efffa3d8700 1 -- 172.21.15.79:0/3807791213 learned_addr learned my addr 172.21.15.79:0/3807791213 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:05.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.926+0000 7efff93d6700 1 --2- 172.21.15.79:0/3807791213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efff40cd410 0x7efff40b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.926+0000 7efff9bd7700 1 --2- 172.21.15.79:0/3807791213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efff40aed90 0x7efff40c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.927+0000 7efffa3d8700 1 -- 172.21.15.79:0/3807791213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efff40aed90 msgr2=0x7efff40c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.927+0000 7efffa3d8700 1 --2- 172.21.15.79:0/3807791213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efff40aed90 0x7efff40c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.927+0000 7efffa3d8700 1 -- 172.21.15.79:0/3807791213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efff40cd410 msgr2=0x7efff40b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:05.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.927+0000 7efffa3d8700 1 --2- 172.21.15.79:0/3807791213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efff40cd410 0x7efff40b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:05.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.927+0000 7efffa3d8700 1 -- 172.21.15.79:0/3807791213 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe40094a0 con 0x7efff415c710 2023-06-23T18:36:05.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.927+0000 7efffa3d8700 1 --2- 172.21.15.79:0/3807791213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efff415c710 0x7efff40b4080 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7effec00cd50 tx=0x7effec00a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:05.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.927+0000 7effea7fc700 1 -- 172.21.15.79:0/3807791213 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec00cea0 con 0x7efff415c710 2023-06-23T18:36:05.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.927+0000 7effea7fc700 1 -- 172.21.15.79:0/3807791213 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec00f480 con 0x7efff415c710 2023-06-23T18:36:05.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.927+0000 7effea7fc700 1 -- 172.21.15.79:0/3807791213 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec00f830 con 0x7efff415c710 2023-06-23T18:36:05.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.927+0000 7efffbe3b700 1 -- 172.21.15.79:0/3807791213 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff40b35a0 con 0x7efff415c710 2023-06-23T18:36:05.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.927+0000 7efffbe3b700 1 -- 172.21.15.79:0/3807791213 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff40c5540 con 0x7efff415c710 2023-06-23T18:36:05.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.928+0000 7efffbe3b700 1 -- 172.21.15.79:0/3807791213 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff406bf30 con 0x7efff415c710 2023-06-23T18:36:05.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.928+0000 7effea7fc700 1 -- 172.21.15.79:0/3807791213 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7effec00b040 con 0x7efff415c710 2023-06-23T18:36:05.938 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.929+0000 7effea7fc700 1 --2- 172.21.15.79:0/3807791213 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7effd806ace0 0x7effd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:05.938 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.929+0000 7effea7fc700 1 -- 172.21.15.79:0/3807791213 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7effec089ae0 con 0x7efff415c710 2023-06-23T18:36:05.938 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.929+0000 7efff9bd7700 1 --2- 172.21.15.79:0/3807791213 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7effd806ace0 0x7effd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:05.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.929+0000 7efff9bd7700 1 --2- 172.21.15.79:0/3807791213 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7effd806ace0 0x7effd806d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7effe40090b0 tx=0x7effe4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:05.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:05.933+0000 7effea7fc700 1 -- 172.21.15.79:0/3807791213 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7effec054c20 con 0x7efff415c710 2023-06-23T18:36:06.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.119+0000 7efffbe3b700 1 -- 172.21.15.79:0/3807791213 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efff40672c0 con 0x7efff415c710 2023-06-23T18:36:06.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.120+0000 7effea7fc700 1 -- 172.21.15.79:0/3807791213 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7effec01d0a0 con 0x7efff415c710 2023-06-23T18:36:06.125 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:06.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.122+0000 7efffbe3b700 1 -- 172.21.15.79:0/3807791213 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7effd806ace0 msgr2=0x7effd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.122+0000 7efffbe3b700 1 --2- 172.21.15.79:0/3807791213 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7effd806ace0 0x7effd806d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7effe40090b0 tx=0x7effe4008030 comp rx=0 tx=0).stop 2023-06-23T18:36:06.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.122+0000 7efffbe3b700 1 -- 172.21.15.79:0/3807791213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efff415c710 msgr2=0x7efff40b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.122+0000 7efffbe3b700 1 --2- 172.21.15.79:0/3807791213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efff415c710 0x7efff40b4080 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7effec00cd50 tx=0x7effec00a560 comp rx=0 tx=0).stop 2023-06-23T18:36:06.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.122+0000 7efffbe3b700 1 -- 172.21.15.79:0/3807791213 shutdown_connections 2023-06-23T18:36:06.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.122+0000 7efffbe3b700 1 --2- 172.21.15.79:0/3807791213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efff415c710 0x7efff40b4080 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.122+0000 7efffbe3b700 1 --2- 172.21.15.79:0/3807791213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efff40aed90 0x7efff40c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.122+0000 7efffbe3b700 1 --2- 172.21.15.79:0/3807791213 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7effd806ace0 0x7effd806d190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.122+0000 7efffbe3b700 1 --2- 172.21.15.79:0/3807791213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efff40cd410 0x7efff40b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.122+0000 7efffbe3b700 1 -- 172.21.15.79:0/3807791213 >> 172.21.15.79:0/3807791213 conn(0x7efff406e200 msgr2=0x7efff406eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:06.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.122+0000 7efffbe3b700 1 -- 172.21.15.79:0/3807791213 shutdown_connections 2023-06-23T18:36:06.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.122+0000 7efffbe3b700 1 -- 172.21.15.79:0/3807791213 wait complete. 2023-06-23T18:36:06.139 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:06.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.257+0000 7f86c60cf700 1 Processor -- start 2023-06-23T18:36:06.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.257+0000 7f86c60cf700 1 -- start start 2023-06-23T18:36:06.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86c60cf700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f86c015d750 0x7f86c015e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86c60cf700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f86c006a5a0 0x7f86c00b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86c60cf700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c0071cf0 con 0x7f86c0072180 2023-06-23T18:36:06.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86c60cf700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c015ecf0 con 0x7f86c006a5a0 2023-06-23T18:36:06.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86c60cf700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c015db20 con 0x7f86c015d750 2023-06-23T18:36:06.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86bffff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f86c006a5a0 0x7f86c00b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86beffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f86c0072180 0x7f86c0069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42784/0 (socket says 172.21.15.79:42784) 2023-06-23T18:36:06.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86bffff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f86c006a5a0 0x7f86c00b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39744/0 (socket says 172.21.15.79:39744) 2023-06-23T18:36:06.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86beffd700 1 -- 172.21.15.79:0/3446840902 learned_addr learned my addr 172.21.15.79:0/3446840902 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:06.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86bf7fe700 1 --2- 172.21.15.79:0/3446840902 >> v2:172.21.15.155:3300/0 conn(0x7f86c015d750 0x7f86c015e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86bffff700 1 -- 172.21.15.79:0/3446840902 >> v2:172.21.15.155:3300/0 conn(0x7f86c015d750 msgr2=0x7f86c015e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:06.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86bffff700 1 --2- 172.21.15.79:0/3446840902 >> v2:172.21.15.155:3300/0 conn(0x7f86c015d750 0x7f86c015e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86bffff700 1 -- 172.21.15.79:0/3446840902 >> v1:172.21.15.79:6789/0 conn(0x7f86c0072180 legacy=0x7f86c0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86be7fc700 1 -- 172.21.15.79:0/3446840902 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610416342 0 0) 0x7f86c0071cf0 con 0x7f86c0072180 2023-06-23T18:36:06.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86bffff700 1 -- 172.21.15.79:0/3446840902 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c00ca310 con 0x7f86c006a5a0 2023-06-23T18:36:06.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86bffff700 1 --2- 172.21.15.79:0/3446840902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c006a5a0 0x7f86c00b12f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86c00cf770 tx=0x7f86b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f951c798eacacf1a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:06.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86be7fc700 1 -- 172.21.15.79:0/3446840902 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b800e070 con 0x7f86c006a5a0 2023-06-23T18:36:06.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.258+0000 7f86be7fc700 1 -- 172.21.15.79:0/3446840902 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b800bbd0 con 0x7f86c006a5a0 2023-06-23T18:36:06.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.259+0000 7f86be7fc700 1 -- 172.21.15.79:0/3446840902 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8012430 con 0x7f86c006a5a0 2023-06-23T18:36:06.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.259+0000 7f86c60cf700 1 -- 172.21.15.79:0/3446840902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c006a5a0 msgr2=0x7f86c00b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.259+0000 7f86c60cf700 1 --2- 172.21.15.79:0/3446840902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c006a5a0 0x7f86c00b12f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86c00cf770 tx=0x7f86b800b330 comp rx=0 tx=0).stop 2023-06-23T18:36:06.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.259+0000 7f86c60cf700 1 -- 172.21.15.79:0/3446840902 shutdown_connections 2023-06-23T18:36:06.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.259+0000 7f86c60cf700 1 --2- 172.21.15.79:0/3446840902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c006a5a0 0x7f86c00b12f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.259+0000 7f86c60cf700 1 --2- 172.21.15.79:0/3446840902 >> v2:172.21.15.155:3300/0 conn(0x7f86c015d750 0x7f86c015e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.259+0000 7f86c60cf700 1 -- 172.21.15.79:0/3446840902 >> 172.21.15.79:0/3446840902 conn(0x7f86c006afa0 msgr2=0x7f86c006b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:06.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.259+0000 7f86c60cf700 1 -- 172.21.15.79:0/3446840902 shutdown_connections 2023-06-23T18:36:06.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.259+0000 7f86c60cf700 1 -- 172.21.15.79:0/3446840902 wait complete. 2023-06-23T18:36:06.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.260+0000 7f86c60cf700 1 Processor -- start 2023-06-23T18:36:06.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.260+0000 7f86c60cf700 1 -- start start 2023-06-23T18:36:06.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.261+0000 7f86c60cf700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86c006a5a0 0x7f86c00c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.261+0000 7f86c60cf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c0072180 0x7f86c014bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.261+0000 7f86c60cf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86c015d750 0x7f86c00c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.261+0000 7f86c60cf700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00c8120 con 0x7f86c0072180 2023-06-23T18:36:06.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.261+0000 7f86c60cf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00c7370 con 0x7f86c015d750 2023-06-23T18:36:06.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.261+0000 7f86c60cf700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00c7650 con 0x7f86c006a5a0 2023-06-23T18:36:06.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.261+0000 7f86beffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c0072180 0x7f86c014bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.261+0000 7f86beffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c0072180 0x7f86c014bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39748/0 (socket says 172.21.15.79:39748) 2023-06-23T18:36:06.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.261+0000 7f86beffd700 1 -- 172.21.15.79:0/1620653796 learned_addr learned my addr 172.21.15.79:0/1620653796 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:06.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.261+0000 7f86bffff700 1 --2- 172.21.15.79:0/1620653796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86c015d750 0x7f86c00c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.262+0000 7f86bf7fe700 1 --2- 172.21.15.79:0/1620653796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86c006a5a0 0x7f86c00c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.262+0000 7f86beffd700 1 -- 172.21.15.79:0/1620653796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86c006a5a0 msgr2=0x7f86c00c2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.262+0000 7f86beffd700 1 --2- 172.21.15.79:0/1620653796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86c006a5a0 0x7f86c00c2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.262+0000 7f86beffd700 1 -- 172.21.15.79:0/1620653796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86c015d750 msgr2=0x7f86c00c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.262+0000 7f86beffd700 1 --2- 172.21.15.79:0/1620653796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86c015d750 0x7f86c00c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.262+0000 7f86beffd700 1 -- 172.21.15.79:0/1620653796 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b8009a70 con 0x7f86c0072180 2023-06-23T18:36:06.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.262+0000 7f86beffd700 1 --2- 172.21.15.79:0/1620653796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c0072180 0x7f86c014bc60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f86b00011e0 tx=0x7f86b000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:06.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.262+0000 7f86b7fff700 1 -- 172.21.15.79:0/1620653796 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b0004c80 con 0x7f86c0072180 2023-06-23T18:36:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.262+0000 7f86b7fff700 1 -- 172.21.15.79:0/1620653796 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b0004e00 con 0x7f86c0072180 2023-06-23T18:36:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.262+0000 7f86c60cf700 1 -- 172.21.15.79:0/1620653796 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c00c8300 con 0x7f86c0072180 2023-06-23T18:36:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.262+0000 7f86b7fff700 1 -- 172.21.15.79:0/1620653796 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b0007310 con 0x7f86c0072180 2023-06-23T18:36:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.262+0000 7f86c60cf700 1 -- 172.21.15.79:0/1620653796 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c014c1a0 con 0x7f86c0072180 2023-06-23T18:36:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.263+0000 7f86c60cf700 1 -- 172.21.15.79:0/1620653796 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c00c7870 con 0x7f86c0072180 2023-06-23T18:36:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.263+0000 7f86b7fff700 1 -- 172.21.15.79:0/1620653796 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f86b0007470 con 0x7f86c0072180 2023-06-23T18:36:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.263+0000 7f86b7fff700 1 --2- 172.21.15.79:0/1620653796 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f86a006ad10 0x7f86a006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.264+0000 7f86b7fff700 1 -- 172.21.15.79:0/1620653796 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f86b00891d0 con 0x7f86c0072180 2023-06-23T18:36:06.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.264+0000 7f86bf7fe700 1 --2- 172.21.15.79:0/1620653796 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f86a006ad10 0x7f86a006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.264+0000 7f86bf7fe700 1 --2- 172.21.15.79:0/1620653796 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f86a006ad10 0x7f86a006d1c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009950 tx=0x7f86a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:06.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.268+0000 7f86b7fff700 1 -- 172.21.15.79:0/1620653796 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f86b0054280 con 0x7f86c0072180 2023-06-23T18:36:06.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.454+0000 7f86c60cf700 1 -- 172.21.15.79:0/1620653796 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f86c006a350 con 0x7f86c0072180 2023-06-23T18:36:06.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.454+0000 7f86b7fff700 1 -- 172.21.15.79:0/1620653796 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f86b00578a0 con 0x7f86c0072180 2023-06-23T18:36:06.460 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:06.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.457+0000 7f86c60cf700 1 -- 172.21.15.79:0/1620653796 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f86a006ad10 msgr2=0x7f86a006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.457+0000 7f86c60cf700 1 --2- 172.21.15.79:0/1620653796 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f86a006ad10 0x7f86a006d1c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009950 tx=0x7f86a8008040 comp rx=0 tx=0).stop 2023-06-23T18:36:06.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.457+0000 7f86c60cf700 1 -- 172.21.15.79:0/1620653796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c0072180 msgr2=0x7f86c014bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.457+0000 7f86c60cf700 1 --2- 172.21.15.79:0/1620653796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c0072180 0x7f86c014bc60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f86b00011e0 tx=0x7f86b000cae0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.457+0000 7f86c60cf700 1 -- 172.21.15.79:0/1620653796 shutdown_connections 2023-06-23T18:36:06.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.457+0000 7f86c60cf700 1 --2- 172.21.15.79:0/1620653796 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f86a006ad10 0x7f86a006d1c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.457+0000 7f86c60cf700 1 --2- 172.21.15.79:0/1620653796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86c015d750 0x7f86c00c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.457+0000 7f86c60cf700 1 --2- 172.21.15.79:0/1620653796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86c006a5a0 0x7f86c00c2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.457+0000 7f86c60cf700 1 --2- 172.21.15.79:0/1620653796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c0072180 0x7f86c014bc60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.457+0000 7f86c60cf700 1 -- 172.21.15.79:0/1620653796 >> 172.21.15.79:0/1620653796 conn(0x7f86c006afa0 msgr2=0x7f86c00cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:06.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.457+0000 7f86c60cf700 1 -- 172.21.15.79:0/1620653796 shutdown_connections 2023-06-23T18:36:06.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.457+0000 7f86c60cf700 1 -- 172.21.15.79:0/1620653796 wait complete. 2023-06-23T18:36:06.473 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:06.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.592+0000 7fc5a1c66700 1 Processor -- start 2023-06-23T18:36:06.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.592+0000 7fc5a1c66700 1 -- start start 2023-06-23T18:36:06.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.592+0000 7fc5a1c66700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc59c06b6b0 0x7fc59c167350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.592+0000 7fc5a1c66700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59c0b0480 con 0x7fc59c0af750 2023-06-23T18:36:06.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.592+0000 7fc5a1c66700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59c0b1a00 con 0x7fc59c15c580 2023-06-23T18:36:06.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.592+0000 7fc5a1c66700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c0cb1b0 con 0x7fc59c06b6b0 2023-06-23T18:36:06.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59affd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fc59c0af750 0x7fc59c06afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42796/0 (socket says 172.21.15.79:42796) 2023-06-23T18:36:06.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59affd700 1 -- 172.21.15.79:0/3206366636 learned_addr learned my addr 172.21.15.79:0/3206366636 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:06.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59bfff700 1 --2- 172.21.15.79:0/3206366636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc59c06b6b0 0x7fc59c167350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 -- 172.21.15.79:0/3206366636 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3172747873 0 0) 0x7fc59c0b0480 con 0x7fc59c0af750 2023-06-23T18:36:06.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 -- 172.21.15.79:0/3206366636 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5840035b0 con 0x7fc59c0af750 2023-06-23T18:36:06.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 -- 172.21.15.79:0/3206366636 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2879865186 0 0) 0x7fc5840035b0 con 0x7fc59c0af750 2023-06-23T18:36:06.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 -- 172.21.15.79:0/3206366636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc59c06b6b0 msgr2=0x7fc59c167350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:06.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 --2- 172.21.15.79:0/3206366636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc59c06b6b0 0x7fc59c167350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 -- 172.21.15.79:0/3206366636 >> v1:172.21.15.155:6789/0 conn(0x7fc59c15c580 legacy=0x7fc59c0b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 -- 172.21.15.79:0/3206366636 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c0afbe0 con 0x7fc59c0af750 2023-06-23T18:36:06.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 -- 172.21.15.79:0/3206366636 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fc590001670 con 0x7fc59c0af750 2023-06-23T18:36:06.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 -- 172.21.15.79:0/3206366636 >> v1:172.21.15.79:6789/0 conn(0x7fc59c0af750 legacy=0x7fc59c06afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 --2- 172.21.15.79:0/3206366636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc584003a30 0x7fc584003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 --2- 172.21.15.79:0/3206366636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc584004850 0x7fc584004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 --2- 172.21.15.79:0/3206366636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc59c06b6b0 0x7fc584005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 -- 172.21.15.79:0/3206366636 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c0cb1b0 con 0x7fc584004850 2023-06-23T18:36:06.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 -- 172.21.15.79:0/3206366636 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58400bb10 con 0x7fc584003a30 2023-06-23T18:36:06.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59a7fc700 1 -- 172.21.15.79:0/3206366636 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58400bcd0 con 0x7fc59c06b6b0 2023-06-23T18:36:06.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59affd700 1 --2- 172.21.15.79:0/3206366636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc584004850 0x7fc584004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59bfff700 1 --2- 172.21.15.79:0/3206366636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc59c06b6b0 0x7fc584005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59b7fe700 1 --2- 172.21.15.79:0/3206366636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc584003a30 0x7fc584003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.606 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59affd700 1 -- 172.21.15.79:0/3206366636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc59c06b6b0 msgr2=0x7fc584005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:06.606 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59affd700 1 --2- 172.21.15.79:0/3206366636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc59c06b6b0 0x7fc584005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.606 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59affd700 1 -- 172.21.15.79:0/3206366636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc584003a30 msgr2=0x7fc584003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:06.606 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59affd700 1 --2- 172.21.15.79:0/3206366636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc584003a30 0x7fc584003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.593+0000 7fc59affd700 1 -- 172.21.15.79:0/3206366636 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5840035b0 con 0x7fc584004850 2023-06-23T18:36:06.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.594+0000 7fc59affd700 1 --2- 172.21.15.79:0/3206366636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc584004850 0x7fc584004cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc59000c210 tx=0x7fc584005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87fe14b45438cc4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:06.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.594+0000 7fc59a7fc700 1 -- 172.21.15.79:0/3206366636 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc590007c10 con 0x7fc584004850 2023-06-23T18:36:06.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.594+0000 7fc59a7fc700 1 -- 172.21.15.79:0/3206366636 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59000cdd0 con 0x7fc584004850 2023-06-23T18:36:06.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.594+0000 7fc59a7fc700 1 -- 172.21.15.79:0/3206366636 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc590005ba0 con 0x7fc584004850 2023-06-23T18:36:06.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.594+0000 7fc5a1c66700 1 -- 172.21.15.79:0/3206366636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc584004850 msgr2=0x7fc584004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.594+0000 7fc5a1c66700 1 --2- 172.21.15.79:0/3206366636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc584004850 0x7fc584004cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc59000c210 tx=0x7fc584005880 comp rx=0 tx=0).stop 2023-06-23T18:36:06.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.594+0000 7fc59bfff700 1 -- 172.21.15.79:0/3206366636 reap_dead start 2023-06-23T18:36:06.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.594+0000 7fc5a1c66700 1 -- 172.21.15.79:0/3206366636 shutdown_connections 2023-06-23T18:36:06.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.594+0000 7fc5a1c66700 1 -- 172.21.15.79:0/3206366636 >> 172.21.15.79:0/3206366636 conn(0x7fc59c099cc0 msgr2=0x7fc59c089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:06.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.594+0000 7fc5a1c66700 1 -- 172.21.15.79:0/3206366636 shutdown_connections 2023-06-23T18:36:06.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.594+0000 7fc5a1c66700 1 -- 172.21.15.79:0/3206366636 wait complete. 2023-06-23T18:36:06.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.595+0000 7fc5a1c66700 1 Processor -- start 2023-06-23T18:36:06.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.596+0000 7fc5a1c66700 1 -- start start 2023-06-23T18:36:06.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.596+0000 7fc5a1c66700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc584004850 0x7fc59c155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.596+0000 7fc5a1c66700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc59c0ae8d0 0x7fc59c1e61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.596+0000 7fc5a1c66700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc59c1e66e0 0x7fc59c1e3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.596+0000 7fc5a1c66700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c1e4d50 con 0x7fc59c1e66e0 2023-06-23T18:36:06.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.596+0000 7fc5a1c66700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c0c9ad0 con 0x7fc584004850 2023-06-23T18:36:06.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.596+0000 7fc5a1c66700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c0c9db0 con 0x7fc59c0ae8d0 2023-06-23T18:36:06.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.596+0000 7fc59bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc59c1e66e0 0x7fc59c1e3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc59bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc59c1e66e0 0x7fc59c1e3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39770/0 (socket says 172.21.15.79:39770) 2023-06-23T18:36:06.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc59bfff700 1 -- 172.21.15.79:0/2927249508 learned_addr learned my addr 172.21.15.79:0/2927249508 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:06.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc59affd700 1 --2- 172.21.15.79:0/2927249508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc59c0ae8d0 0x7fc59c1e61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc59b7fe700 1 --2- 172.21.15.79:0/2927249508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc584004850 0x7fc59c155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc59bfff700 1 -- 172.21.15.79:0/2927249508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc59c0ae8d0 msgr2=0x7fc59c1e61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc59bfff700 1 --2- 172.21.15.79:0/2927249508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc59c0ae8d0 0x7fc59c1e61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc59bfff700 1 -- 172.21.15.79:0/2927249508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc584004850 msgr2=0x7fc59c155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc59bfff700 1 --2- 172.21.15.79:0/2927249508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc584004850 0x7fc59c155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc59bfff700 1 -- 172.21.15.79:0/2927249508 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c0ae0d0 con 0x7fc59c1e66e0 2023-06-23T18:36:06.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc59bfff700 1 --2- 172.21.15.79:0/2927249508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc59c1e66e0 0x7fc59c1e3f00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0725b0 tx=0x7fc594011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:06.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc583fff700 1 -- 172.21.15.79:0/2927249508 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59400e040 con 0x7fc59c1e66e0 2023-06-23T18:36:06.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc583fff700 1 -- 172.21.15.79:0/2927249508 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59400f040 con 0x7fc59c1e66e0 2023-06-23T18:36:06.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc5a1c66700 1 -- 172.21.15.79:0/2927249508 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59c0ae3b0 con 0x7fc59c1e66e0 2023-06-23T18:36:06.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc583fff700 1 -- 172.21.15.79:0/2927249508 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59400fe40 con 0x7fc59c1e66e0 2023-06-23T18:36:06.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.597+0000 7fc5a1c66700 1 -- 172.21.15.79:0/2927249508 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59c1e4440 con 0x7fc59c1e66e0 2023-06-23T18:36:06.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.598+0000 7fc5a1c66700 1 -- 172.21.15.79:0/2927249508 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59c0aef20 con 0x7fc59c1e66e0 2023-06-23T18:36:06.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.599+0000 7fc583fff700 1 -- 172.21.15.79:0/2927249508 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc594015430 con 0x7fc59c1e66e0 2023-06-23T18:36:06.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.599+0000 7fc583fff700 1 --2- 172.21.15.79:0/2927249508 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc58806ace0 0x7fc58806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.599+0000 7fc583fff700 1 -- 172.21.15.79:0/2927249508 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc594089bf0 con 0x7fc59c1e66e0 2023-06-23T18:36:06.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.599+0000 7fc59b7fe700 1 --2- 172.21.15.79:0/2927249508 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc58806ace0 0x7fc58806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.599+0000 7fc59b7fe700 1 --2- 172.21.15.79:0/2927249508 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc58806ace0 0x7fc58806d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc59c1f5870 tx=0x7fc58c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:06.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.603+0000 7fc583fff700 1 -- 172.21.15.79:0/2927249508 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc59c0aef20 con 0x7fc59c1e66e0 2023-06-23T18:36:06.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.791+0000 7fc5a1c66700 1 -- 172.21.15.79:0/2927249508 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc59c0aef20 con 0x7fc59c1e66e0 2023-06-23T18:36:06.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.791+0000 7fc583fff700 1 -- 172.21.15.79:0/2927249508 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc59c0aef20 con 0x7fc59c1e66e0 2023-06-23T18:36:06.796 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:06.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.793+0000 7fc5a1c66700 1 -- 172.21.15.79:0/2927249508 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc58806ace0 msgr2=0x7fc58806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.793+0000 7fc5a1c66700 1 --2- 172.21.15.79:0/2927249508 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc58806ace0 0x7fc58806d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc59c1f5870 tx=0x7fc58c009040 comp rx=0 tx=0).stop 2023-06-23T18:36:06.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.793+0000 7fc5a1c66700 1 -- 172.21.15.79:0/2927249508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc59c1e66e0 msgr2=0x7fc59c1e3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.793+0000 7fc5a1c66700 1 --2- 172.21.15.79:0/2927249508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc59c1e66e0 0x7fc59c1e3f00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0725b0 tx=0x7fc594011f60 comp rx=0 tx=0).stop 2023-06-23T18:36:06.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.793+0000 7fc5a1c66700 1 -- 172.21.15.79:0/2927249508 shutdown_connections 2023-06-23T18:36:06.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.793+0000 7fc5a1c66700 1 --2- 172.21.15.79:0/2927249508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc59c1e66e0 0x7fc59c1e3f00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.793+0000 7fc5a1c66700 1 --2- 172.21.15.79:0/2927249508 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc58806ace0 0x7fc58806d190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.793+0000 7fc5a1c66700 1 --2- 172.21.15.79:0/2927249508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc584004850 0x7fc59c155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.793+0000 7fc5a1c66700 1 --2- 172.21.15.79:0/2927249508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc59c0ae8d0 0x7fc59c1e61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.793+0000 7fc5a1c66700 1 -- 172.21.15.79:0/2927249508 >> 172.21.15.79:0/2927249508 conn(0x7fc59c099cc0 msgr2=0x7fc59c0cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:06.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.793+0000 7fc5a1c66700 1 -- 172.21.15.79:0/2927249508 shutdown_connections 2023-06-23T18:36:06.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.793+0000 7fc5a1c66700 1 -- 172.21.15.79:0/2927249508 wait complete. 2023-06-23T18:36:06.810 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:36:06.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.925+0000 7f8015d2c700 1 Processor -- start 2023-06-23T18:36:06.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.925+0000 7f8015d2c700 1 -- start start 2023-06-23T18:36:06.932 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.925+0000 7f8015d2c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8010071490 0x7f8010071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.932 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.925+0000 7f8015d2c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f80100b1270 0x7f8010071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.932 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.925+0000 7f8015d2c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f80100aee30 0x7f8010069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.932 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.925+0000 7f8015d2c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80101608f0 con 0x7f80100b1270 2023-06-23T18:36:06.932 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.925+0000 7f8015d2c700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80100722e0 con 0x7f80100aee30 2023-06-23T18:36:06.932 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.925+0000 7f8015d2c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8010160e20 con 0x7f8010071490 2023-06-23T18:36:06.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.925+0000 7f800effd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f80100b1270 0x7f8010071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.926+0000 7f800effd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f80100b1270 0x7f8010071da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39776/0 (socket says 172.21.15.79:39776) 2023-06-23T18:36:06.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.926+0000 7f800effd700 1 -- 172.21.15.79:0/2630606023 learned_addr learned my addr 172.21.15.79:0/2630606023 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:06.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.926+0000 7f800f7fe700 1 --2- 172.21.15.79:0/2630606023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8010071490 0x7f8010071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.926+0000 7f800ffff700 1 --2- 172.21.15.79:0/2630606023 >> v2:172.21.15.155:3300/0 conn(0x7f80100aee30 0x7f8010069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.926+0000 7f800effd700 1 -- 172.21.15.79:0/2630606023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8010071490 msgr2=0x7f8010071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:06.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.926+0000 7f800effd700 1 --2- 172.21.15.79:0/2630606023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8010071490 0x7f8010071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.926+0000 7f800effd700 1 -- 172.21.15.79:0/2630606023 >> v2:172.21.15.155:3300/0 conn(0x7f80100aee30 msgr2=0x7f8010069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:06.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.926+0000 7f800effd700 1 --2- 172.21.15.79:0/2630606023 >> v2:172.21.15.155:3300/0 conn(0x7f80100aee30 0x7f8010069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.926+0000 7f800effd700 1 -- 172.21.15.79:0/2630606023 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80100cd4f0 con 0x7f80100b1270 2023-06-23T18:36:06.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.926+0000 7f800effd700 1 --2- 172.21.15.79:0/2630606023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80100b1270 0x7f8010071da0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f80100cea10 tx=0x7f8000009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5b9c5dff7fa9dc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:06.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.926+0000 7f800e7fc700 1 -- 172.21.15.79:0/2630606023 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000011830 con 0x7f80100b1270 2023-06-23T18:36:06.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.926+0000 7f800e7fc700 1 -- 172.21.15.79:0/2630606023 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8000011990 con 0x7f80100b1270 2023-06-23T18:36:06.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.926+0000 7f800e7fc700 1 -- 172.21.15.79:0/2630606023 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000011d20 con 0x7f80100b1270 2023-06-23T18:36:06.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.927+0000 7f8015d2c700 1 -- 172.21.15.79:0/2630606023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80100b1270 msgr2=0x7f8010071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.927+0000 7f8015d2c700 1 --2- 172.21.15.79:0/2630606023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80100b1270 0x7f8010071da0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f80100cea10 tx=0x7f8000009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.927+0000 7f8015d2c700 1 -- 172.21.15.79:0/2630606023 shutdown_connections 2023-06-23T18:36:06.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.927+0000 7f8015d2c700 1 --2- 172.21.15.79:0/2630606023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8010071490 0x7f8010071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.927+0000 7f8015d2c700 1 --2- 172.21.15.79:0/2630606023 >> v2:172.21.15.155:3300/0 conn(0x7f80100aee30 0x7f8010069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.927+0000 7f8015d2c700 1 --2- 172.21.15.79:0/2630606023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80100b1270 0x7f8010071da0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.927+0000 7f8015d2c700 1 -- 172.21.15.79:0/2630606023 >> 172.21.15.79:0/2630606023 conn(0x7f801006d660 msgr2=0x7f801006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:06.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.927+0000 7f8015d2c700 1 -- 172.21.15.79:0/2630606023 shutdown_connections 2023-06-23T18:36:06.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.927+0000 7f8015d2c700 1 -- 172.21.15.79:0/2630606023 wait complete. 2023-06-23T18:36:06.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.928+0000 7f8015d2c700 1 Processor -- start 2023-06-23T18:36:06.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.928+0000 7f8015d2c700 1 -- start start 2023-06-23T18:36:06.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f8015d2c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8010071490 0x7f80101537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f8015d2c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80100aee30 0x7f80100b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f8015d2c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80100b1270 0x7f80100c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f8015d2c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80100c4fb0 con 0x7f80100aee30 2023-06-23T18:36:06.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f8015d2c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80100b5510 con 0x7f80100b1270 2023-06-23T18:36:06.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f8015d2c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80100ab950 con 0x7f8010071490 2023-06-23T18:36:06.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f800effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80100aee30 0x7f80100b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f800effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80100aee30 0x7f80100b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39782/0 (socket says 172.21.15.79:39782) 2023-06-23T18:36:06.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f800effd700 1 -- 172.21.15.79:0/2718394141 learned_addr learned my addr 172.21.15.79:0/2718394141 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:06.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f800f7fe700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8010071490 0x7f80101537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f800ffff700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80100b1270 0x7f80100c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f800effd700 1 -- 172.21.15.79:0/2718394141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8010071490 msgr2=0x7f80101537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f800effd700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8010071490 0x7f80101537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f800effd700 1 -- 172.21.15.79:0/2718394141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80100b1270 msgr2=0x7f80100c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:06.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f800effd700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80100b1270 0x7f80100c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:06.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f800effd700 1 -- 172.21.15.79:0/2718394141 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8000009550 con 0x7f80100aee30 2023-06-23T18:36:06.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f800f7fe700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8010071490 0x7f80101537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:06.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f800ffff700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80100b1270 0x7f80100c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:06.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f800effd700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80100aee30 0x7f80100b4fd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8000014040 tx=0x7f8000004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:06.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f8007fff700 1 -- 172.21.15.79:0/2718394141 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000004510 con 0x7f80100aee30 2023-06-23T18:36:06.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f8007fff700 1 -- 172.21.15.79:0/2718394141 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800001e070 con 0x7f80100aee30 2023-06-23T18:36:06.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f8015d2c700 1 -- 172.21.15.79:0/2718394141 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80100abc30 con 0x7f80100aee30 2023-06-23T18:36:06.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.929+0000 7f8007fff700 1 -- 172.21.15.79:0/2718394141 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000038e0 con 0x7f80100aee30 2023-06-23T18:36:06.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.930+0000 7f8015d2c700 1 -- 172.21.15.79:0/2718394141 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8010153f60 con 0x7f80100aee30 2023-06-23T18:36:06.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.930+0000 7f8015d2c700 1 -- 172.21.15.79:0/2718394141 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801004e240 con 0x7f80100aee30 2023-06-23T18:36:06.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.931+0000 7f8007fff700 1 -- 172.21.15.79:0/2718394141 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8000003a40 con 0x7f80100aee30 2023-06-23T18:36:06.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.931+0000 7f8007fff700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7ff006ace0 0x7f7ff006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:06.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.931+0000 7f8007fff700 1 -- 172.21.15.79:0/2718394141 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f800008a2c0 con 0x7f80100aee30 2023-06-23T18:36:06.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.931+0000 7f800f7fe700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7ff006ace0 0x7f7ff006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:06.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.932+0000 7f800f7fe700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7ff006ace0 0x7f7ff006d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f80100c8d00 tx=0x7f7ff8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:06.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:06.935+0000 7f8007fff700 1 -- 172.21.15.79:0/2718394141 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8000055400 con 0x7f80100aee30 2023-06-23T18:36:07.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.170+0000 7f8015d2c700 1 -- 172.21.15.79:0/2718394141 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80100c4d80 con 0x7f80100aee30 2023-06-23T18:36:07.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.170+0000 7f8007fff700 1 -- 172.21.15.79:0/2718394141 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f800002c3b0 con 0x7f80100aee30 2023-06-23T18:36:07.177 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:07.177 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/399719238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":399719238},{"type":"v1","addr":"172.21.15.155:6834","nonce":399719238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1846440916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1846440916},{"type":"v1","addr":"172.21.15.155:6835","nonce":1846440916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3310180303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3310180303},{"type":"v1","addr":"172.21.15.79:6835","nonce":3310180303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:35:53.511495+0000","modified":"2023-06-23T18:35:56.527609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6837/3900137670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3900137670},{"type":"v1","addr":"172.21.15.79:6837","nonce":3900137670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:36:07.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.172+0000 7f8015d2c700 1 -- 172.21.15.79:0/2718394141 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7ff006ace0 msgr2=0x7f7ff006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:07.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.172+0000 7f8015d2c700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7ff006ace0 0x7f7ff006d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f80100c8d00 tx=0x7f7ff8008040 comp rx=0 tx=0).stop 2023-06-23T18:36:07.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.172+0000 7f8015d2c700 1 -- 172.21.15.79:0/2718394141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80100aee30 msgr2=0x7f80100b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:07.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.172+0000 7f8015d2c700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80100aee30 0x7f80100b4fd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8000014040 tx=0x7f8000004750 comp rx=0 tx=0).stop 2023-06-23T18:36:07.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.172+0000 7f8015d2c700 1 -- 172.21.15.79:0/2718394141 shutdown_connections 2023-06-23T18:36:07.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.172+0000 7f8015d2c700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7ff006ace0 0x7f7ff006d190 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.172+0000 7f8015d2c700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80100b1270 0x7f80100c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.172+0000 7f8015d2c700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8010071490 0x7f80101537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.172+0000 7f8015d2c700 1 --2- 172.21.15.79:0/2718394141 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80100aee30 0x7f80100b4fd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.172+0000 7f8015d2c700 1 -- 172.21.15.79:0/2718394141 >> 172.21.15.79:0/2718394141 conn(0x7f801006d660 msgr2=0x7f80100cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:07.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.172+0000 7f8015d2c700 1 -- 172.21.15.79:0/2718394141 shutdown_connections 2023-06-23T18:36:07.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.172+0000 7f8015d2c700 1 -- 172.21.15.79:0/2718394141 wait complete. 2023-06-23T18:36:07.180 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 8 2023-06-23T18:36:07.189 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:07.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.304+0000 7f5c4d13a700 1 Processor -- start 2023-06-23T18:36:07.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.304+0000 7f5c4d13a700 1 -- start start 2023-06-23T18:36:07.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c4d13a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c480b1f40 0x7f5c48069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c4d13a700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5c4806a3d0 0x7f5c480cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c4d13a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c480ca550 con 0x7f5c480b4f20 2023-06-23T18:36:07.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c4d13a700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4806a850 con 0x7f5c4806a3d0 2023-06-23T18:36:07.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c4d13a700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4815a810 con 0x7f5c480b1f40 2023-06-23T18:36:07.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c46d9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5c480b4f20 0x7f5c480c9e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42830/0 (socket says 172.21.15.79:42830) 2023-06-23T18:36:07.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c46d9d700 1 -- 172.21.15.79:0/1408407408 learned_addr learned my addr 172.21.15.79:0/1408407408 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:07.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c4659c700 1 --2- 172.21.15.79:0/1408407408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c480b1f40 0x7f5c48069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 -- 172.21.15.79:0/1408407408 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915852559 0 0) 0x7f5c480ca550 con 0x7f5c480b4f20 2023-06-23T18:36:07.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c4759e700 1 --2- 172.21.15.79:0/1408407408 >> v2:172.21.15.155:3300/0 conn(0x7f5c4806a3d0 0x7f5c480cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 -- 172.21.15.79:0/1408407408 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c300035b0 con 0x7f5c480b4f20 2023-06-23T18:36:07.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 -- 172.21.15.79:0/1408407408 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 982701624 0 0) 0x7f5c300035b0 con 0x7f5c480b4f20 2023-06-23T18:36:07.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 -- 172.21.15.79:0/1408407408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c480b1f40 msgr2=0x7f5c48069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:07.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 --2- 172.21.15.79:0/1408407408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c480b1f40 0x7f5c48069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 -- 172.21.15.79:0/1408407408 >> v2:172.21.15.155:3300/0 conn(0x7f5c4806a3d0 msgr2=0x7f5c480cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:07.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 --2- 172.21.15.79:0/1408407408 >> v2:172.21.15.155:3300/0 conn(0x7f5c4806a3d0 0x7f5c480cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 -- 172.21.15.79:0/1408407408 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c480b52f0 con 0x7f5c480b4f20 2023-06-23T18:36:07.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 -- 172.21.15.79:0/1408407408 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f5c38001670 con 0x7f5c480b4f20 2023-06-23T18:36:07.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 -- 172.21.15.79:0/1408407408 >> v1:172.21.15.79:6789/0 conn(0x7f5c480b4f20 legacy=0x7f5c480c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:07.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 --2- 172.21.15.79:0/1408407408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c30003a00 0x7f5c30003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 --2- 172.21.15.79:0/1408407408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c480b1f40 0x7f5c30004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 --2- 172.21.15.79:0/1408407408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c30004d90 0x7f5c30005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 -- 172.21.15.79:0/1408407408 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4806a850 con 0x7f5c30003a00 2023-06-23T18:36:07.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 -- 172.21.15.79:0/1408407408 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4815a810 con 0x7f5c30004d90 2023-06-23T18:36:07.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c45d9b700 1 -- 172.21.15.79:0/1408407408 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3000ba10 con 0x7f5c480b1f40 2023-06-23T18:36:07.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c46d9d700 1 --2- 172.21.15.79:0/1408407408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c30003a00 0x7f5c30003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c4659c700 1 --2- 172.21.15.79:0/1408407408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c480b1f40 0x7f5c30004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.305+0000 7f5c4759e700 1 --2- 172.21.15.79:0/1408407408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c30004d90 0x7f5c30005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c46d9d700 1 -- 172.21.15.79:0/1408407408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c480b1f40 msgr2=0x7f5c30004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:07.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c46d9d700 1 --2- 172.21.15.79:0/1408407408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c480b1f40 0x7f5c30004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c46d9d700 1 -- 172.21.15.79:0/1408407408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c30004d90 msgr2=0x7f5c30005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:07.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c46d9d700 1 --2- 172.21.15.79:0/1408407408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c30004d90 0x7f5c30005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c46d9d700 1 -- 172.21.15.79:0/1408407408 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c300035b0 con 0x7f5c30003a00 2023-06-23T18:36:07.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c4659c700 1 --2- 172.21.15.79:0/1408407408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c480b1f40 0x7f5c30004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:07.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c4759e700 1 --2- 172.21.15.79:0/1408407408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c30004d90 0x7f5c30005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:07.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c46d9d700 1 --2- 172.21.15.79:0/1408407408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c30003a00 0x7f5c30003e10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800c320 tx=0x7f5c30005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f247782b9b2ac9ba server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:07.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c45d9b700 1 -- 172.21.15.79:0/1408407408 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38007c40 con 0x7f5c30003a00 2023-06-23T18:36:07.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c45d9b700 1 -- 172.21.15.79:0/1408407408 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3800cdc0 con 0x7f5c30003a00 2023-06-23T18:36:07.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c45d9b700 1 -- 172.21.15.79:0/1408407408 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38005ba0 con 0x7f5c30003a00 2023-06-23T18:36:07.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1408407408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c30003a00 msgr2=0x7f5c30003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:07.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c4d13a700 1 --2- 172.21.15.79:0/1408407408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c30003a00 0x7f5c30003e10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800c320 tx=0x7f5c30005880 comp rx=0 tx=0).stop 2023-06-23T18:36:07.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c4759e700 1 -- 172.21.15.79:0/1408407408 reap_dead start 2023-06-23T18:36:07.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1408407408 shutdown_connections 2023-06-23T18:36:07.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1408407408 >> 172.21.15.79:0/1408407408 conn(0x7f5c4806d660 msgr2=0x7f5c4806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:07.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1408407408 shutdown_connections 2023-06-23T18:36:07.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.306+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1408407408 wait complete. 2023-06-23T18:36:07.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.308+0000 7f5c4d13a700 1 Processor -- start 2023-06-23T18:36:07.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.308+0000 7f5c4d13a700 1 -- start start 2023-06-23T18:36:07.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4d13a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c30003a00 0x7f5c48155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4d13a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c48151590 0x7f5c481e2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4d13a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c48158bc0 0x7f5c48159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4d13a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c48159580 con 0x7f5c48158bc0 2023-06-23T18:36:07.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4d13a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c480c60f0 con 0x7f5c30003a00 2023-06-23T18:36:07.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4d13a700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c481521f0 con 0x7f5c48151590 2023-06-23T18:36:07.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4759e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c48158bc0 0x7f5c48159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4759e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c48158bc0 0x7f5c48159040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39804/0 (socket says 172.21.15.79:39804) 2023-06-23T18:36:07.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4759e700 1 -- 172.21.15.79:0/1653733545 learned_addr learned my addr 172.21.15.79:0/1653733545 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:07.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c46d9d700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c30003a00 0x7f5c48155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4659c700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c48151590 0x7f5c481e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4759e700 1 -- 172.21.15.79:0/1653733545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c48151590 msgr2=0x7f5c481e2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:07.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4759e700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c48151590 0x7f5c481e2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4759e700 1 -- 172.21.15.79:0/1653733545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c30003a00 msgr2=0x7f5c48155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:07.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4759e700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c30003a00 0x7f5c48155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4759e700 1 -- 172.21.15.79:0/1653733545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c481e2ac0 con 0x7f5c48158bc0 2023-06-23T18:36:07.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4659c700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c48151590 0x7f5c481e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:07.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4759e700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c48158bc0 0x7f5c48159040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5c4806d250 tx=0x7f5c400104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:07.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c2f7fe700 1 -- 172.21.15.79:0/1653733545 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4001b070 con 0x7f5c48158bc0 2023-06-23T18:36:07.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c2f7fe700 1 -- 172.21.15.79:0/1653733545 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4000f040 con 0x7f5c48158bc0 2023-06-23T18:36:07.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1653733545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c48152440 con 0x7f5c48158bc0 2023-06-23T18:36:07.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c2f7fe700 1 -- 172.21.15.79:0/1653733545 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4000e050 con 0x7f5c48158bc0 2023-06-23T18:36:07.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.309+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1653733545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c48152a00 con 0x7f5c48158bc0 2023-06-23T18:36:07.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.310+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1653733545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4806e3d0 con 0x7f5c48158bc0 2023-06-23T18:36:07.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.311+0000 7f5c2f7fe700 1 -- 172.21.15.79:0/1653733545 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c40010d80 con 0x7f5c48158bc0 2023-06-23T18:36:07.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.311+0000 7f5c2f7fe700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5c3406ace0 0x7f5c3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.311+0000 7f5c2f7fe700 1 -- 172.21.15.79:0/1653733545 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c400893d0 con 0x7f5c48158bc0 2023-06-23T18:36:07.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.311+0000 7f5c46d9d700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5c3406ace0 0x7f5c3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.312+0000 7f5c46d9d700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5c3406ace0 0x7f5c3406d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5c38008fd0 tx=0x7f5c3800a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:07.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.315+0000 7f5c2f7fe700 1 -- 172.21.15.79:0/1653733545 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5c40054480 con 0x7f5c48158bc0 2023-06-23T18:36:07.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.502+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1653733545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c480b4040 con 0x7f5c48158bc0 2023-06-23T18:36:07.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.503+0000 7f5c2f7fe700 1 -- 172.21.15.79:0/1653733545 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7f5c40007ea0 con 0x7f5c48158bc0 2023-06-23T18:36:07.508 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:07.508 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:07.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.505+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1653733545 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5c3406ace0 msgr2=0x7f5c3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:07.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.505+0000 7f5c4d13a700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5c3406ace0 0x7f5c3406d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5c38008fd0 tx=0x7f5c3800a000 comp rx=0 tx=0).stop 2023-06-23T18:36:07.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.505+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1653733545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c48158bc0 msgr2=0x7f5c48159040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:07.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.505+0000 7f5c4d13a700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c48158bc0 0x7f5c48159040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5c4806d250 tx=0x7f5c400104a0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.505+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1653733545 shutdown_connections 2023-06-23T18:36:07.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.505+0000 7f5c4d13a700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c48158bc0 0x7f5c48159040 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.505+0000 7f5c4d13a700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5c3406ace0 0x7f5c3406d190 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.505+0000 7f5c4d13a700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c30003a00 0x7f5c48155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.505+0000 7f5c4d13a700 1 --2- 172.21.15.79:0/1653733545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c48151590 0x7f5c481e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.505+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1653733545 >> 172.21.15.79:0/1653733545 conn(0x7f5c4806d660 msgr2=0x7f5c480ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:07.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.505+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1653733545 shutdown_connections 2023-06-23T18:36:07.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.505+0000 7f5c4d13a700 1 -- 172.21.15.79:0/1653733545 wait complete. 2023-06-23T18:36:07.522 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:36:07.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.636+0000 7f74cba26700 1 Processor -- start 2023-06-23T18:36:07.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.637+0000 7f74cba26700 1 -- start start 2023-06-23T18:36:07.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.637+0000 7f74cba26700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f74c40b1270 0x7f74c415c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.637+0000 7f74cba26700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f74c40aee30 0x7f74c415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.637+0000 7f74cba26700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c41608f0 con 0x7f74c4071490 2023-06-23T18:36:07.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.637+0000 7f74cba26700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c4160e20 con 0x7f74c40aee30 2023-06-23T18:36:07.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.637+0000 7f74cba26700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c40cd4f0 con 0x7f74c40b1270 2023-06-23T18:36:07.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.637+0000 7f74c9fc3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f74c40aee30 0x7f74c415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.638+0000 7f74c9fc3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f74c40aee30 0x7f74c415cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39812/0 (socket says 172.21.15.79:39812) 2023-06-23T18:36:07.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.638+0000 7f74c9fc3700 1 -- 172.21.15.79:0/2488919479 learned_addr learned my addr 172.21.15.79:0/2488919479 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:07.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.638+0000 7f74c8fc1700 1 --2- 172.21.15.79:0/2488919479 >> v2:172.21.15.155:3300/0 conn(0x7f74c40b1270 0x7f74c415c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.638+0000 7f74c9fc3700 1 -- 172.21.15.79:0/2488919479 >> v2:172.21.15.155:3300/0 conn(0x7f74c40b1270 msgr2=0x7f74c415c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:07.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.638+0000 7f74c9fc3700 1 --2- 172.21.15.79:0/2488919479 >> v2:172.21.15.155:3300/0 conn(0x7f74c40b1270 0x7f74c415c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.638+0000 7f74c9fc3700 1 -- 172.21.15.79:0/2488919479 >> v1:172.21.15.155:6789/0 conn(0x7f74c4071490 legacy=0x7f74c4071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:07.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.638+0000 7f74c9fc3700 1 -- 172.21.15.79:0/2488919479 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c4160b40 con 0x7f74c40aee30 2023-06-23T18:36:07.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.638+0000 7f74c9fc3700 1 --2- 172.21.15.79:0/2488919479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74c40aee30 0x7f74c415cf30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f74c40cd230 tx=0x7f74bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec046475f8b13057 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:07.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.638+0000 7f74b7fff700 1 -- 172.21.15.79:0/2488919479 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc00e070 con 0x7f74c40aee30 2023-06-23T18:36:07.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.638+0000 7f74b7fff700 1 -- 172.21.15.79:0/2488919479 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc00b910 con 0x7f74c40aee30 2023-06-23T18:36:07.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.638+0000 7f74b7fff700 1 -- 172.21.15.79:0/2488919479 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc00bcc0 con 0x7f74c40aee30 2023-06-23T18:36:07.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.639+0000 7f74cba26700 1 -- 172.21.15.79:0/2488919479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74c40aee30 msgr2=0x7f74c415cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:07.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.639+0000 7f74cba26700 1 --2- 172.21.15.79:0/2488919479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74c40aee30 0x7f74c415cf30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f74c40cd230 tx=0x7f74bc00b330 comp rx=0 tx=0).stop 2023-06-23T18:36:07.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.639+0000 7f74cba26700 1 -- 172.21.15.79:0/2488919479 shutdown_connections 2023-06-23T18:36:07.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.639+0000 7f74cba26700 1 --2- 172.21.15.79:0/2488919479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74c40aee30 0x7f74c415cf30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.639+0000 7f74cba26700 1 --2- 172.21.15.79:0/2488919479 >> v2:172.21.15.155:3300/0 conn(0x7f74c40b1270 0x7f74c415c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.639+0000 7f74cba26700 1 -- 172.21.15.79:0/2488919479 >> 172.21.15.79:0/2488919479 conn(0x7f74c406d660 msgr2=0x7f74c406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:07.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.639+0000 7f74cba26700 1 -- 172.21.15.79:0/2488919479 shutdown_connections 2023-06-23T18:36:07.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.639+0000 7f74cba26700 1 -- 172.21.15.79:0/2488919479 wait complete. 2023-06-23T18:36:07.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.640+0000 7f74cba26700 1 Processor -- start 2023-06-23T18:36:07.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.640+0000 7f74cba26700 1 -- start start 2023-06-23T18:36:07.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74cba26700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74c4071490 0x7f74c40c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74cba26700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74c40aee30 0x7f74c4153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74cba26700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74c40b1270 0x7f74c4154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74cba26700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c4154850 con 0x7f74c4071490 2023-06-23T18:36:07.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74cba26700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40c9850 con 0x7f74c40aee30 2023-06-23T18:36:07.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74cba26700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40b3070 con 0x7f74c40b1270 2023-06-23T18:36:07.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74c97c2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74c4071490 0x7f74c40c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74c97c2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74c4071490 0x7f74c40c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39814/0 (socket says 172.21.15.79:39814) 2023-06-23T18:36:07.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74c97c2700 1 -- 172.21.15.79:0/2772908239 learned_addr learned my addr 172.21.15.79:0/2772908239 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:07.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74c8fc1700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74c40aee30 0x7f74c4153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74c9fc3700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74c40b1270 0x7f74c4154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74c97c2700 1 -- 172.21.15.79:0/2772908239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74c40b1270 msgr2=0x7f74c4154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:07.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74c97c2700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74c40b1270 0x7f74c4154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74c97c2700 1 -- 172.21.15.79:0/2772908239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74c40aee30 msgr2=0x7f74c4153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:07.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74c97c2700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74c40aee30 0x7f74c4153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74c97c2700 1 -- 172.21.15.79:0/2772908239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc009b30 con 0x7f74c4071490 2023-06-23T18:36:07.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74c8fc1700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74c40aee30 0x7f74c4153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:07.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74c9fc3700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74c40b1270 0x7f74c4154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:07.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74c97c2700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74c4071490 0x7f74c40c6c80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f74b800c450 tx=0x7f74b800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:07.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74b5ffb700 1 -- 172.21.15.79:0/2772908239 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b80052d0 con 0x7f74c4071490 2023-06-23T18:36:07.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74b5ffb700 1 -- 172.21.15.79:0/2772908239 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b8005430 con 0x7f74c4071490 2023-06-23T18:36:07.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74cba26700 1 -- 172.21.15.79:0/2772908239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c40b3350 con 0x7f74c4071490 2023-06-23T18:36:07.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74b5ffb700 1 -- 172.21.15.79:0/2772908239 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b80038c0 con 0x7f74c4071490 2023-06-23T18:36:07.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.641+0000 7f74cba26700 1 -- 172.21.15.79:0/2772908239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c40b3910 con 0x7f74c4071490 2023-06-23T18:36:07.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.642+0000 7f74cba26700 1 -- 172.21.15.79:0/2772908239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c40b07c0 con 0x7f74c4071490 2023-06-23T18:36:07.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.643+0000 7f74b5ffb700 1 -- 172.21.15.79:0/2772908239 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f74b8003a20 con 0x7f74c4071490 2023-06-23T18:36:07.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.643+0000 7f74b5ffb700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f74a806ace0 0x7f74a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:07.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.643+0000 7f74b5ffb700 1 -- 172.21.15.79:0/2772908239 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f74b80886f0 con 0x7f74c4071490 2023-06-23T18:36:07.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.643+0000 7f74c8fc1700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f74a806ace0 0x7f74a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:07.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.644+0000 7f74c8fc1700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f74a806ace0 0x7f74a806d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f74c40aeab0 tx=0x7f74c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:07.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.647+0000 7f74b5ffb700 1 -- 172.21.15.79:0/2772908239 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f74b8053830 con 0x7f74c4071490 2023-06-23T18:36:07.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.876+0000 7f74cba26700 1 -- 172.21.15.79:0/2772908239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74c415b900 con 0x7f74c4071490 2023-06-23T18:36:07.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.877+0000 7f74b5ffb700 1 -- 172.21.15.79:0/2772908239 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f74b8056e50 con 0x7f74c4071490 2023-06-23T18:36:07.882 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:07.882 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/399719238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":399719238},{"type":"v1","addr":"172.21.15.155:6834","nonce":399719238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1846440916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1846440916},{"type":"v1","addr":"172.21.15.155:6835","nonce":1846440916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3310180303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3310180303},{"type":"v1","addr":"172.21.15.79:6835","nonce":3310180303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:35:53.511495+0000","modified":"2023-06-23T18:35:56.527609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6837/3900137670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3900137670},{"type":"v1","addr":"172.21.15.79:6837","nonce":3900137670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:36:07.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.879+0000 7f74cba26700 1 -- 172.21.15.79:0/2772908239 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f74a806ace0 msgr2=0x7f74a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:07.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.879+0000 7f74cba26700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f74a806ace0 0x7f74a806d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f74c40aeab0 tx=0x7f74c0008040 comp rx=0 tx=0).stop 2023-06-23T18:36:07.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.879+0000 7f74cba26700 1 -- 172.21.15.79:0/2772908239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74c4071490 msgr2=0x7f74c40c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:07.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.879+0000 7f74cba26700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74c4071490 0x7f74c40c6c80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f74b800c450 tx=0x7f74b800c810 comp rx=0 tx=0).stop 2023-06-23T18:36:07.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.879+0000 7f74cba26700 1 -- 172.21.15.79:0/2772908239 shutdown_connections 2023-06-23T18:36:07.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.879+0000 7f74cba26700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74c40b1270 0x7f74c4154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.879+0000 7f74cba26700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74c4071490 0x7f74c40c6c80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.879+0000 7f74cba26700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f74a806ace0 0x7f74a806d190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.879+0000 7f74cba26700 1 --2- 172.21.15.79:0/2772908239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74c40aee30 0x7f74c4153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:07.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.879+0000 7f74cba26700 1 -- 172.21.15.79:0/2772908239 >> 172.21.15.79:0/2772908239 conn(0x7f74c406d660 msgr2=0x7f74c415a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:07.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.879+0000 7f74cba26700 1 -- 172.21.15.79:0/2772908239 shutdown_connections 2023-06-23T18:36:07.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:07.879+0000 7f74cba26700 1 -- 172.21.15.79:0/2772908239 wait complete. 2023-06-23T18:36:07.886 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 8 2023-06-23T18:36:08.298 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-06-23T18:35:53.511495+0000', 'modified': '2023-06-23T18:35:56.527609+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.79:6837/3900137670', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6836', 'nonce': 3900137670}, {'type': 'v1', 'addr': '172.21.15.79:6837', 'nonce': 3900137670}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-06-23T18:36:08.298 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-06-23T18:36:08.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-06-23T18:36:08.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.417+0000 7f52e0a88700 1 Processor -- start 2023-06-23T18:36:08.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.417+0000 7f52e0a88700 1 -- start start 2023-06-23T18:36:08.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52e0a88700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc0b1270 0x7f52dc15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52e0a88700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f52dc0aee30 0x7f52dc15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52e0a88700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52dc1608f0 con 0x7f52dc071490 2023-06-23T18:36:08.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52e0a88700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52dc160e20 con 0x7f52dc0aee30 2023-06-23T18:36:08.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52e0a88700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc0cd4f0 con 0x7f52dc0b1270 2023-06-23T18:36:08.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52dad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f52dc0aee30 0x7f52dc15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52dad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f52dc0aee30 0x7f52dc15cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39826/0 (socket says 172.21.15.79:39826) 2023-06-23T18:36:08.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52dad9d700 1 -- 172.21.15.79:0/3252739130 learned_addr learned my addr 172.21.15.79:0/3252739130 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:08.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52d9d9b700 1 --2- 172.21.15.79:0/3252739130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc0b1270 0x7f52dc15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52dad9d700 1 -- 172.21.15.79:0/3252739130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc0b1270 msgr2=0x7f52dc15c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:08.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52dad9d700 1 --2- 172.21.15.79:0/3252739130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc0b1270 0x7f52dc15c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52dad9d700 1 -- 172.21.15.79:0/3252739130 >> v1:172.21.15.155:6789/0 conn(0x7f52dc071490 legacy=0x7f52dc071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:08.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52dad9d700 1 -- 172.21.15.79:0/3252739130 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc160b40 con 0x7f52dc0aee30 2023-06-23T18:36:08.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52dad9d700 1 --2- 172.21.15.79:0/3252739130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52dc0aee30 0x7f52dc15cf30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009e50 tx=0x7f52d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afae6f110b46194 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:08.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52d959a700 1 -- 172.21.15.79:0/3252739130 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400e070 con 0x7f52dc0aee30 2023-06-23T18:36:08.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52d959a700 1 -- 172.21.15.79:0/3252739130 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d400b9c0 con 0x7f52dc0aee30 2023-06-23T18:36:08.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.418+0000 7f52d959a700 1 -- 172.21.15.79:0/3252739130 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400bd70 con 0x7f52dc0aee30 2023-06-23T18:36:08.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.419+0000 7f52e0a88700 1 -- 172.21.15.79:0/3252739130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52dc0aee30 msgr2=0x7f52dc15cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:08.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.419+0000 7f52e0a88700 1 --2- 172.21.15.79:0/3252739130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52dc0aee30 0x7f52dc15cf30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009e50 tx=0x7f52d400b330 comp rx=0 tx=0).stop 2023-06-23T18:36:08.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.419+0000 7f52e0a88700 1 -- 172.21.15.79:0/3252739130 shutdown_connections 2023-06-23T18:36:08.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.419+0000 7f52e0a88700 1 --2- 172.21.15.79:0/3252739130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52dc0aee30 0x7f52dc15cf30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.419+0000 7f52e0a88700 1 --2- 172.21.15.79:0/3252739130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc0b1270 0x7f52dc15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.419+0000 7f52e0a88700 1 -- 172.21.15.79:0/3252739130 >> 172.21.15.79:0/3252739130 conn(0x7f52dc06d660 msgr2=0x7f52dc06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:08.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.419+0000 7f52e0a88700 1 -- 172.21.15.79:0/3252739130 shutdown_connections 2023-06-23T18:36:08.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.419+0000 7f52e0a88700 1 -- 172.21.15.79:0/3252739130 wait complete. 2023-06-23T18:36:08.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.420+0000 7f52e0a88700 1 Processor -- start 2023-06-23T18:36:08.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.420+0000 7f52e0a88700 1 -- start start 2023-06-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.421+0000 7f52e0a88700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52dc071490 0x7f52dc159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.421+0000 7f52e0a88700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc0aee30 0x7f52dc0b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.421+0000 7f52e0a88700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52dc0b1270 0x7f52dc0b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.421+0000 7f52e0a88700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc0b3580 con 0x7f52dc071490 2023-06-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.421+0000 7f52e0a88700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc159540 con 0x7f52dc0b1270 2023-06-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.421+0000 7f52e0a88700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc0b2970 con 0x7f52dc0aee30 2023-06-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.421+0000 7f52da59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52dc071490 0x7f52dc159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.421+0000 7f52da59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52dc071490 0x7f52dc159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39828/0 (socket says 172.21.15.79:39828) 2023-06-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.421+0000 7f52da59c700 1 -- 172.21.15.79:0/60953163 learned_addr learned my addr 172.21.15.79:0/60953163 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.421+0000 7f52d9d9b700 1 --2- 172.21.15.79:0/60953163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc0aee30 0x7f52dc0b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.421+0000 7f52dad9d700 1 --2- 172.21.15.79:0/60953163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52dc0b1270 0x7f52dc0b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.421+0000 7f52da59c700 1 -- 172.21.15.79:0/60953163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc0aee30 msgr2=0x7f52dc0b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.421+0000 7f52da59c700 1 --2- 172.21.15.79:0/60953163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc0aee30 0x7f52dc0b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.422+0000 7f52da59c700 1 -- 172.21.15.79:0/60953163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52dc0b1270 msgr2=0x7f52dc0b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:08.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.422+0000 7f52da59c700 1 --2- 172.21.15.79:0/60953163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52dc0b1270 0x7f52dc0b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.422+0000 7f52da59c700 1 -- 172.21.15.79:0/60953163 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d4009b30 con 0x7f52dc071490 2023-06-23T18:36:08.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.422+0000 7f52da59c700 1 --2- 172.21.15.79:0/60953163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52dc071490 0x7f52dc159000 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00c450 tx=0x7f52cc00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:08.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.422+0000 7f52caffd700 1 -- 172.21.15.79:0/60953163 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0052d0 con 0x7f52dc071490 2023-06-23T18:36:08.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.422+0000 7f52caffd700 1 -- 172.21.15.79:0/60953163 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52cc005430 con 0x7f52dc071490 2023-06-23T18:36:08.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.422+0000 7f52e0a88700 1 -- 172.21.15.79:0/60953163 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc0b2c50 con 0x7f52dc071490 2023-06-23T18:36:08.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.422+0000 7f52caffd700 1 -- 172.21.15.79:0/60953163 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0038c0 con 0x7f52dc071490 2023-06-23T18:36:08.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.422+0000 7f52e0a88700 1 -- 172.21.15.79:0/60953163 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc0b3210 con 0x7f52dc071490 2023-06-23T18:36:08.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.423+0000 7f52caffd700 1 -- 172.21.15.79:0/60953163 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f52cc003a20 con 0x7f52dc071490 2023-06-23T18:36:08.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.423+0000 7f52caffd700 1 --2- 172.21.15.79:0/60953163 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f52bc06ace0 0x7f52bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.424+0000 7f52caffd700 1 -- 172.21.15.79:0/60953163 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52cc0887a0 con 0x7f52dc071490 2023-06-23T18:36:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.424+0000 7f52d9d9b700 1 --2- 172.21.15.79:0/60953163 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f52bc06ace0 0x7f52bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.424+0000 7f52d9d9b700 1 --2- 172.21.15.79:0/60953163 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f52bc06ace0 0x7f52bc06d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f52dc1f59b0 tx=0x7f52d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.426+0000 7f52ecc0db80 1 Processor -- start 2023-06-23T18:36:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.426+0000 7f52ecc0db80 1 -- start start 2023-06-23T18:36:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52ecc0db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a340bf50 0x5612a340c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52ecc0db80 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a3433d80 0x5612a3432820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52ecc0db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 0x5612a3436440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52ecc0db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612a3310c60 con 0x5612a340bf50 2023-06-23T18:36:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52ecc0db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612a33639f0 con 0x5612a3409740 2023-06-23T18:36:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52ecc0db80 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612a335ce80 con 0x5612a3433d80 2023-06-23T18:36:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52da59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a340bf50 0x5612a340c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52da59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a340bf50 0x5612a340c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39836/0 (socket says 172.21.15.79:39836) 2023-06-23T18:36:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52da59c700 1 -- 172.21.15.79:0/2310430106 learned_addr learned my addr 172.21.15.79:0/2310430106 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:08.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52d9d9b700 1 --2- 172.21.15.79:0/2310430106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a3433d80 0x5612a3432820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52da59c700 1 -- 172.21.15.79:0/2310430106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a3433d80 msgr2=0x5612a3432820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:08.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52da59c700 1 --2- 172.21.15.79:0/2310430106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a3433d80 0x5612a3432820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52dad9d700 1 --2- 172.21.15.79:0/2310430106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 0x5612a3436440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52da59c700 1 -- 172.21.15.79:0/2310430106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 msgr2=0x5612a3436440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:08.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52da59c700 1 --2- 172.21.15.79:0/2310430106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 0x5612a3436440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52da59c700 1 -- 172.21.15.79:0/2310430106 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612a31de5e0 con 0x5612a340bf50 2023-06-23T18:36:08.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52dad9d700 1 --2- 172.21.15.79:0/2310430106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 0x5612a3436440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:08.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52d9d9b700 1 --2- 172.21.15.79:0/2310430106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a3433d80 0x5612a3432820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:08.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52da59c700 1 --2- 172.21.15.79:0/2310430106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a340bf50 0x5612a340c320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f52cc086b90 tx=0x7f52cc056e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897e4af6dee068fa server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:08.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52bbfff700 1 -- 172.21.15.79:0/2310430106 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc05b4d0 con 0x5612a340bf50 2023-06-23T18:36:08.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52bbfff700 1 -- 172.21.15.79:0/2310430106 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52cc05f860 con 0x5612a340bf50 2023-06-23T18:36:08.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.427+0000 7f52bbfff700 1 -- 172.21.15.79:0/2310430106 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc05b170 con 0x5612a340bf50 2023-06-23T18:36:08.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.428+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2310430106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a340bf50 msgr2=0x5612a340c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:08.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.428+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2310430106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a340bf50 0x5612a340c320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f52cc086b90 tx=0x7f52cc056e70 comp rx=0 tx=0).stop 2023-06-23T18:36:08.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.428+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2310430106 shutdown_connections 2023-06-23T18:36:08.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.428+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2310430106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 0x5612a3436440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.428+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2310430106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a340bf50 0x5612a340c320 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.428+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2310430106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a3433d80 0x5612a3432820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.428+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2310430106 >> 172.21.15.79:0/2310430106 conn(0x5612a3400fc0 msgr2=0x5612a34511a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:08.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.428+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2310430106 shutdown_connections 2023-06-23T18:36:08.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.428+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2310430106 wait complete. 2023-06-23T18:36:08.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.429+0000 7f52ecc0db80 1 -- start start 2023-06-23T18:36:08.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.429+0000 7f52ecc0db80 1 Processor -- start 2023-06-23T18:36:08.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.429+0000 7f52ecc0db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 0x5612a3452e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.429+0000 7f52ecc0db80 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a340bf50 0x5612a3454f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.429+0000 7f52ecc0db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a3433d80 0x5612a342c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52ecc0db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612a32fca70 con 0x5612a3433d80 2023-06-23T18:36:08.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52ecc0db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612a32fc770 con 0x5612a3409740 2023-06-23T18:36:08.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52ecc0db80 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612a31e08d0 con 0x5612a340bf50 2023-06-23T18:36:08.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52ecc0db80 10 client.0 _ll_register_callbacks cb 0x5612a3302210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:36:08.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52dad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a3433d80 0x5612a342c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52dad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a3433d80 0x5612a342c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39846/0 (socket says 172.21.15.79:39846) 2023-06-23T18:36:08.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52dad9d700 1 -- 172.21.15.79:0/2977311727 learned_addr learned my addr 172.21.15.79:0/2977311727 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:08.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52da59c700 1 --2- 172.21.15.79:0/2977311727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 0x5612a3452e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52d9d9b700 1 --2- 172.21.15.79:0/2977311727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a340bf50 0x5612a3454f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52dad9d700 1 -- 172.21.15.79:0/2977311727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a340bf50 msgr2=0x5612a3454f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:08.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52dad9d700 1 --2- 172.21.15.79:0/2977311727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a340bf50 0x5612a3454f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52dad9d700 1 -- 172.21.15.79:0/2977311727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 msgr2=0x5612a3452e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:08.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52dad9d700 1 --2- 172.21.15.79:0/2977311727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 0x5612a3452e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52dad9d700 1 -- 172.21.15.79:0/2977311727 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc0b3210 con 0x5612a3433d80 2023-06-23T18:36:08.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52dad9d700 1 --2- 172.21.15.79:0/2977311727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a3433d80 0x5612a342c450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f52d4003c50 tx=0x7f52d4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534e8a32456d2626 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:08.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2023-06-23T18:36:08.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52b97fa700 1 -- 172.21.15.79:0/2977311727 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4004450 con 0x5612a3433d80 2023-06-23T18:36:08.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2977311727 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612a33fe780 con 0x5612a3433d80 2023-06-23T18:36:08.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52b97fa700 1 -- 172.21.15.79:0/2977311727 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52d40045b0 con 0x5612a3433d80 2023-06-23T18:36:08.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52b97fa700 1 -- 172.21.15.79:0/2977311727 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52d4003e60 con 0x5612a3433d80 2023-06-23T18:36:08.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52ecc0db80 10 client.4644 fetch_fsmap learned FSMap version 8 2023-06-23T18:36:08.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2977311727 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5612a318ad90 con 0x5612a3433d80 2023-06-23T18:36:08.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.430+0000 7f52b97fa700 1 -- 172.21.15.79:0/2977311727 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5612a33fe780 con 0x5612a3433d80 2023-06-23T18:36:08.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.431+0000 7f52b97fa700 1 -- 172.21.15.79:0/2977311727 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f52d400b050 con 0x5612a3433d80 2023-06-23T18:36:08.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.431+0000 7f52ecc0db80 10 client.4644 fetch_fsmap finished waiting for FSMap version 8 2023-06-23T18:36:08.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.431+0000 7f52ecc0db80 10 client.4644 resolve_mds: resolved name 'a' to daemon mds.a 2023-06-23T18:36:08.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.431+0000 7f52ecc0db80 20 client.4644 populate_metadata read hostname 'smithi079' 2023-06-23T18:36:08.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.431+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2977311727 >> [v2:172.21.15.79:6836/3900137670,v1:172.21.15.79:6837/3900137670] conn(0x5612a3405240 0x5612a3405610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.431+0000 7f52ecc0db80 4 client.4644 mds_command: new command op to 4263 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:36:08.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.431+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2977311727 --> [v2:172.21.15.79:6836/3900137670,v1:172.21.15.79:6837/3900137670] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612a3355e60 con 0x5612a3405240 2023-06-23T18:36:08.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.431+0000 7f52da59c700 1 --2- 172.21.15.79:0/2977311727 >> [v2:172.21.15.79:6836/3900137670,v1:172.21.15.79:6837/3900137670] conn(0x5612a3405240 0x5612a3405610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.431+0000 7f52b97fa700 10 client.4644 ms_handle_connect on v2:172.21.15.79:6836/3900137670 2023-06-23T18:36:08.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.431+0000 7f52da59c700 1 --2- 172.21.15.79:0/2977311727 >> [v2:172.21.15.79:6836/3900137670,v1:172.21.15.79:6837/3900137670] conn(0x5612a3405240 0x5612a3405610 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=670fa7dacd6d069e server_cookie=80cda742780b6ebd in_seq=0 out_seq=0 2023-06-23T18:36:08.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.431+0000 7f52b97fa700 1 -- 172.21.15.79:0/2977311727 <== mds.0 v2:172.21.15.79:6836/3900137670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5612a3355e60 con 0x5612a3405240 2023-06-23T18:36:08.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.431+0000 7f52b97fa700 10 client.4644 handle_command_reply: tid=0 2023-06-23T18:36:08.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.431+0000 7f52ecc0db80 1 client.4644 shutdown 2023-06-23T18:36:08.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.432+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2977311727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a3433d80 msgr2=0x5612a342c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:08.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.432+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2977311727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a3433d80 0x5612a342c450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f52d4003c50 tx=0x7f52d4004b40 comp rx=0 tx=0).stop 2023-06-23T18:36:08.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.432+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2977311727 shutdown_connections 2023-06-23T18:36:08.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.432+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2977311727 >> [v2:172.21.15.79:6836/3900137670,v1:172.21.15.79:6837/3900137670] conn(0x5612a3405240 0x5612a3405610 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.432+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2977311727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a3433d80 0x5612a342c450 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.432+0000 7f52b97fa700 0 client.4644 ms_handle_reset on v2:172.21.15.79:6836/3900137670 2023-06-23T18:36:08.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.432+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2977311727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 0x5612a3452e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.432+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2977311727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a340bf50 0x5612a3454f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.432+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2977311727 >> 172.21.15.79:0/2977311727 conn(0x5612a3400fc0 msgr2=0x5612a3434180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:08.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.432+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2977311727 shutdown_connections 2023-06-23T18:36:08.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.432+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2977311727 wait complete. 2023-06-23T18:36:08.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.432+0000 7f52ecc0db80 20 client.4644 trim_cache size 0 max 16384 2023-06-23T18:36:08.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.449+0000 7f52ecc0db80 1 Processor -- start 2023-06-23T18:36:08.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.449+0000 7f52ecc0db80 1 -- start start 2023-06-23T18:36:08.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52ecc0db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 0x5612a3452620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52ecc0db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a340bf50 0x5612a34359c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52ecc0db80 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a3433d80 0x5612a340b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52ecc0db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612a34329f0 con 0x5612a340bf50 2023-06-23T18:36:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52ecc0db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612a3460f20 con 0x5612a3409740 2023-06-23T18:36:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52ecc0db80 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612a3405610 con 0x5612a3433d80 2023-06-23T18:36:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52d9d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a340bf50 0x5612a34359c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52d9d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a340bf50 0x5612a34359c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39850/0 (socket says 172.21.15.79:39850) 2023-06-23T18:36:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52d9d9b700 1 -- 172.21.15.79:0/1540037828 learned_addr learned my addr 172.21.15.79:0/1540037828 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:08.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52dad9d700 1 --2- 172.21.15.79:0/1540037828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a3433d80 0x5612a340b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52da59c700 1 --2- 172.21.15.79:0/1540037828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 0x5612a3452620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52d9d9b700 1 -- 172.21.15.79:0/1540037828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a3433d80 msgr2=0x5612a340b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:08.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52d9d9b700 1 --2- 172.21.15.79:0/1540037828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a3433d80 0x5612a340b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52d9d9b700 1 -- 172.21.15.79:0/1540037828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 msgr2=0x5612a3452620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:08.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52d9d9b700 1 --2- 172.21.15.79:0/1540037828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 0x5612a3452620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.450+0000 7f52d9d9b700 1 -- 172.21.15.79:0/1540037828 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612a3173c10 con 0x5612a340bf50 2023-06-23T18:36:08.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.451+0000 7f52da59c700 1 --2- 172.21.15.79:0/1540037828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 0x5612a3452620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:08.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.451+0000 7f52d9d9b700 1 --2- 172.21.15.79:0/1540037828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a340bf50 0x5612a34359c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5612a3408010 tx=0x7f52d0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eed4aba5dd56ac9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:08.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.451+0000 7f52c8ff9700 1 -- 172.21.15.79:0/1540037828 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00156c0 con 0x5612a340bf50 2023-06-23T18:36:08.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.451+0000 7f52c8ff9700 1 -- 172.21.15.79:0/1540037828 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d0015820 con 0x5612a340bf50 2023-06-23T18:36:08.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.451+0000 7f52c8ff9700 1 -- 172.21.15.79:0/1540037828 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0015bb0 con 0x5612a340bf50 2023-06-23T18:36:08.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.451+0000 7f52ecc0db80 1 -- 172.21.15.79:0/1540037828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a340bf50 msgr2=0x5612a34359c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:08.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.451+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/1540037828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a340bf50 0x5612a34359c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5612a3408010 tx=0x7f52d0014000 comp rx=0 tx=0).stop 2023-06-23T18:36:08.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.451+0000 7f52ecc0db80 1 -- 172.21.15.79:0/1540037828 shutdown_connections 2023-06-23T18:36:08.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.451+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/1540037828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a3433d80 0x5612a340b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.451+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/1540037828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3409740 0x5612a3452620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.451+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/1540037828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a340bf50 0x5612a34359c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.451+0000 7f52ecc0db80 1 -- 172.21.15.79:0/1540037828 >> 172.21.15.79:0/1540037828 conn(0x5612a3400fc0 msgr2=0x5612a3473c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:08.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.451+0000 7f52ecc0db80 1 -- 172.21.15.79:0/1540037828 shutdown_connections 2023-06-23T18:36:08.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.451+0000 7f52ecc0db80 1 -- 172.21.15.79:0/1540037828 wait complete. 2023-06-23T18:36:08.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.452+0000 7f52ecc0db80 1 -- start start 2023-06-23T18:36:08.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.452+0000 7f52ecc0db80 1 Processor -- start 2023-06-23T18:36:08.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52ecc0db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a3409740 0x5612a349fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52ecc0db80 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a340bf50 0x5612a344ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52ecc0db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3433d80 0x5612a344d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52ecc0db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612a344e060 con 0x5612a3409740 2023-06-23T18:36:08.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52ecc0db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612a345eee0 con 0x5612a3433d80 2023-06-23T18:36:08.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52ecc0db80 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612a344a880 con 0x5612a340bf50 2023-06-23T18:36:08.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52ecc0db80 10 client.0 _ll_register_callbacks cb 0x5612a3302210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:36:08.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52da59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a3409740 0x5612a349fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52da59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a3409740 0x5612a349fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39856/0 (socket says 172.21.15.79:39856) 2023-06-23T18:36:08.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52da59c700 1 -- 172.21.15.79:0/2997049610 learned_addr learned my addr 172.21.15.79:0/2997049610 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:08.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52dad9d700 1 --2- 172.21.15.79:0/2997049610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3433d80 0x5612a344d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52d9d9b700 1 --2- 172.21.15.79:0/2997049610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a340bf50 0x5612a344ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52da59c700 1 -- 172.21.15.79:0/2997049610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a340bf50 msgr2=0x5612a344ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:08.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52da59c700 1 --2- 172.21.15.79:0/2997049610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a340bf50 0x5612a344ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52da59c700 1 -- 172.21.15.79:0/2997049610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3433d80 msgr2=0x5612a344d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:08.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52da59c700 1 --2- 172.21.15.79:0/2997049610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3433d80 0x5612a344d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52da59c700 1 -- 172.21.15.79:0/2997049610 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d000e040 con 0x5612a3409740 2023-06-23T18:36:08.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52d9d9b700 1 --2- 172.21.15.79:0/2997049610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a340bf50 0x5612a344ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:08.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52da59c700 1 --2- 172.21.15.79:0/2997049610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a3409740 0x5612a349fc50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f52cc053a10 tx=0x7f52cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aec48419147bae2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:08.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2023-06-23T18:36:08.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52b97fa700 1 -- 172.21.15.79:0/2997049610 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc055710 con 0x5612a3409740 2023-06-23T18:36:08.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52b97fa700 1 -- 172.21.15.79:0/2997049610 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52cc05f3d0 con 0x5612a3409740 2023-06-23T18:36:08.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.453+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2997049610 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52cc05f860 con 0x5612a3409740 2023-06-23T18:36:08.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52b97fa700 1 -- 172.21.15.79:0/2997049610 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52cc05f590 con 0x5612a3409740 2023-06-23T18:36:08.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52ecc0db80 10 client.4650 fetch_fsmap learned FSMap version 8 2023-06-23T18:36:08.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2997049610 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5612a31b1200 con 0x5612a3409740 2023-06-23T18:36:08.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52b97fa700 1 -- 172.21.15.79:0/2997049610 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52cc05f860 con 0x5612a3409740 2023-06-23T18:36:08.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52b97fa700 1 -- 172.21.15.79:0/2997049610 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f52cc05d0b0 con 0x5612a3409740 2023-06-23T18:36:08.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52ecc0db80 10 client.4650 fetch_fsmap finished waiting for FSMap version 8 2023-06-23T18:36:08.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52ecc0db80 10 client.4650 resolve_mds: resolved name 'a' to daemon mds.a 2023-06-23T18:36:08.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52ecc0db80 20 client.4650 populate_metadata read hostname 'smithi079' 2023-06-23T18:36:08.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2997049610 >> [v2:172.21.15.79:6836/3900137670,v1:172.21.15.79:6837/3900137670] conn(0x5612a3434650 0x5612a340b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52ecc0db80 4 client.4650 mds_command: new command op to 4263 tid=0[{"prefix": "status"}] 2023-06-23T18:36:08.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2997049610 --> [v2:172.21.15.79:6836/3900137670,v1:172.21.15.79:6837/3900137670] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5612a3318000 con 0x5612a3434650 2023-06-23T18:36:08.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52d9d9b700 1 --2- 172.21.15.79:0/2997049610 >> [v2:172.21.15.79:6836/3900137670,v1:172.21.15.79:6837/3900137670] conn(0x5612a3434650 0x5612a340b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52b97fa700 10 client.4650 ms_handle_connect on v2:172.21.15.79:6836/3900137670 2023-06-23T18:36:08.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52d9d9b700 1 --2- 172.21.15.79:0/2997049610 >> [v2:172.21.15.79:6836/3900137670,v1:172.21.15.79:6837/3900137670] conn(0x5612a3434650 0x5612a340b0f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=865559348153fc1 server_cookie=5eb70d4f0cbaef3c in_seq=0 out_seq=0 2023-06-23T18:36:08.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52b97fa700 1 -- 172.21.15.79:0/2997049610 <== mds.0 v2:172.21.15.79:6836/3900137670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5612a3318000 con 0x5612a3434650 2023-06-23T18:36:08.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52b97fa700 10 client.4650 handle_command_reply: tid=0 2023-06-23T18:36:08.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.454+0000 7f52ecc0db80 1 client.4650 shutdown 2023-06-23T18:36:08.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2997049610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a3409740 msgr2=0x5612a349fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:08.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2997049610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a3409740 0x5612a349fc50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f52cc053a10 tx=0x7f52cc017000 comp rx=0 tx=0).stop 2023-06-23T18:36:08.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2997049610 shutdown_connections 2023-06-23T18:36:08.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2997049610 >> [v2:172.21.15.79:6836/3900137670,v1:172.21.15.79:6837/3900137670] conn(0x5612a3434650 0x5612a340b0f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2997049610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612a3433d80 0x5612a344d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52b97fa700 0 client.4650 ms_handle_reset on v2:172.21.15.79:6836/3900137670 2023-06-23T18:36:08.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2997049610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612a3409740 0x5612a349fc50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52ecc0db80 1 --2- 172.21.15.79:0/2997049610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612a340bf50 0x5612a344ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2997049610 >> 172.21.15.79:0/2997049610 conn(0x5612a3400fc0 msgr2=0x5612a344ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:08.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2997049610 shutdown_connections 2023-06-23T18:36:08.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52ecc0db80 1 -- 172.21.15.79:0/2997049610 wait complete. 2023-06-23T18:36:08.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52ecc0db80 20 client.4650 trim_cache size 0 max 16384 2023-06-23T18:36:08.475 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-23T18:36:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "c6d43825-8b0e-40e8-8d7f-a5ab36592233", 2023-06-23T18:36:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 0, 2023-06-23T18:36:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 4263, 2023-06-23T18:36:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2023-06-23T18:36:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2023-06-23T18:36:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2023-06-23T18:36:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 14.942507145, 2023-06-23T18:36:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 8, 2023-06-23T18:36:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 20, 2023-06-23T18:36:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 20, 2023-06-23T18:36:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 20.368001968000002 2023-06-23T18:36:08.477 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-23T18:36:08.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52e0a88700 1 -- 172.21.15.79:0/60953163 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f52bc06ace0 msgr2=0x7f52bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:08.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52e0a88700 1 --2- 172.21.15.79:0/60953163 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f52bc06ace0 0x7f52bc06d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f52dc1f59b0 tx=0x7f52d0008040 comp rx=0 tx=0).stop 2023-06-23T18:36:08.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52e0a88700 1 -- 172.21.15.79:0/60953163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52dc071490 msgr2=0x7f52dc159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:08.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52e0a88700 1 --2- 172.21.15.79:0/60953163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52dc071490 0x7f52dc159000 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00c450 tx=0x7f52cc00c760 comp rx=0 tx=0).stop 2023-06-23T18:36:08.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52e0a88700 1 -- 172.21.15.79:0/60953163 shutdown_connections 2023-06-23T18:36:08.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52e0a88700 1 --2- 172.21.15.79:0/60953163 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f52bc06ace0 0x7f52bc06d190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52e0a88700 1 --2- 172.21.15.79:0/60953163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52dc0b1270 0x7f52dc0b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52e0a88700 1 --2- 172.21.15.79:0/60953163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52dc071490 0x7f52dc159000 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52e0a88700 1 --2- 172.21.15.79:0/60953163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc0aee30 0x7f52dc0b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52e0a88700 1 -- 172.21.15.79:0/60953163 >> 172.21.15.79:0/60953163 conn(0x7f52dc06d660 msgr2=0x7f52dc071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:08.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52e0a88700 1 -- 172.21.15.79:0/60953163 shutdown_connections 2023-06-23T18:36:08.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.455+0000 7f52e0a88700 1 -- 172.21.15.79:0/60953163 wait complete. 2023-06-23T18:36:08.480 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-06-23T18:36:08.506 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-06-23T18:36:08.506 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-06-23T18:36:08.506 INFO:tasks.ceph_fuse:client.0 config is {} 2023-06-23T18:36:08.507 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-23T18:36:08.508 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-23T18:36:08.508 INFO:tasks.ceph_fuse:client.2 config is {} 2023-06-23T18:36:08.508 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-23T18:36:08.509 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-06-23T18:36:08.509 INFO:tasks.ceph_fuse:client.1 config is {} 2023-06-23T18:36:08.509 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-23T18:36:08.509 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-06-23T18:36:08.509 INFO:tasks.ceph_fuse:client.3 config is {} 2023-06-23T18:36:08.510 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-23T18:36:08.510 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-06-23T18:36:08.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:08.510 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2023-06-23T18:36:08.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:08.529 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link delete ceph-brx 2023-06-23T18:36:08.598 INFO:teuthology.orchestra.run.smithi079.stderr:Cannot find device "ceph-brx" 2023-06-23T18:36:08.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:36:08.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:08.600 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-06-23T18:36:08.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:08.622 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link delete ceph-brx 2023-06-23T18:36:08.697 INFO:teuthology.orchestra.run.smithi155.stderr:Cannot find device "ceph-brx" 2023-06-23T18:36:08.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:36:08.699 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-06-23T18:36:08.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-23T18:36:08.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.816+0000 7fdb45eb2700 1 Processor -- start 2023-06-23T18:36:08.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.816+0000 7fdb45eb2700 1 -- start start 2023-06-23T18:36:08.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.816+0000 7fdb45eb2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdb40159170 0x7fdb400cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.816+0000 7fdb45eb2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb400ce7f0 0x7fdb400cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.816+0000 7fdb45eb2700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4015cd50 con 0x7fdb400cd900 2023-06-23T18:36:08.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.816+0000 7fdb45eb2700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb400cdd80 con 0x7fdb40159170 2023-06-23T18:36:08.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.816+0000 7fdb45eb2700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4015cfa0 con 0x7fdb400ce7f0 2023-06-23T18:36:08.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.816+0000 7fdb3f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdb40159170 0x7fdb400cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.817+0000 7fdb3f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdb40159170 0x7fdb400cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39864/0 (socket says 172.21.15.79:39864) 2023-06-23T18:36:08.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.817+0000 7fdb3f7fe700 1 -- 172.21.15.79:0/4283568971 learned_addr learned my addr 172.21.15.79:0/4283568971 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:08.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.817+0000 7fdb3ffff700 1 --2- 172.21.15.79:0/4283568971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb400ce7f0 0x7fdb400cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.817+0000 7fdb3f7fe700 1 -- 172.21.15.79:0/4283568971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb400ce7f0 msgr2=0x7fdb400cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:08.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.817+0000 7fdb3f7fe700 1 --2- 172.21.15.79:0/4283568971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb400ce7f0 0x7fdb400cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.817+0000 7fdb3f7fe700 1 -- 172.21.15.79:0/4283568971 >> v1:172.21.15.155:6789/0 conn(0x7fdb400cd900 legacy=0x7fdb400ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:08.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.817+0000 7fdb3f7fe700 1 -- 172.21.15.79:0/4283568971 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4015d280 con 0x7fdb40159170 2023-06-23T18:36:08.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.817+0000 7fdb3f7fe700 1 --2- 172.21.15.79:0/4283568971 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb40159170 0x7fdb400cd3c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdb300098f0 tx=0x7fdb30009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8263a3baadc015e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:08.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.817+0000 7fdb3e7fc700 1 -- 172.21.15.79:0/4283568971 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30004230 con 0x7fdb40159170 2023-06-23T18:36:08.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.817+0000 7fdb3e7fc700 1 -- 172.21.15.79:0/4283568971 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb30004390 con 0x7fdb40159170 2023-06-23T18:36:08.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.817+0000 7fdb3e7fc700 1 -- 172.21.15.79:0/4283568971 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb300036a0 con 0x7fdb40159170 2023-06-23T18:36:08.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.817+0000 7fdb45eb2700 1 -- 172.21.15.79:0/4283568971 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb40159170 msgr2=0x7fdb400cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:08.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.817+0000 7fdb45eb2700 1 --2- 172.21.15.79:0/4283568971 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb40159170 0x7fdb400cd3c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdb300098f0 tx=0x7fdb30009c00 comp rx=0 tx=0).stop 2023-06-23T18:36:08.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.818+0000 7fdb45eb2700 1 -- 172.21.15.79:0/4283568971 shutdown_connections 2023-06-23T18:36:08.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.818+0000 7fdb45eb2700 1 --2- 172.21.15.79:0/4283568971 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb40159170 0x7fdb400cd3c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.818+0000 7fdb45eb2700 1 --2- 172.21.15.79:0/4283568971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb400ce7f0 0x7fdb400cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.818+0000 7fdb45eb2700 1 -- 172.21.15.79:0/4283568971 >> 172.21.15.79:0/4283568971 conn(0x7fdb4006d660 msgr2=0x7fdb4006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:08.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.818+0000 7fdb45eb2700 1 -- 172.21.15.79:0/4283568971 shutdown_connections 2023-06-23T18:36:08.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.818+0000 7fdb45eb2700 1 -- 172.21.15.79:0/4283568971 wait complete. 2023-06-23T18:36:08.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.819+0000 7fdb45eb2700 1 Processor -- start 2023-06-23T18:36:08.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.819+0000 7fdb45eb2700 1 -- start start 2023-06-23T18:36:08.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb45eb2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb400cd900 0x7fdb40157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb45eb2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb400ce7f0 0x7fdb401e39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb45eb2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb40159170 0x7fdb401e3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb45eb2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb401e4630 con 0x7fdb400cd900 2023-06-23T18:36:08.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb45eb2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb401e4e90 con 0x7fdb40159170 2023-06-23T18:36:08.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb45eb2700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb400b4320 con 0x7fdb400ce7f0 2023-06-23T18:36:08.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb3f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb400cd900 0x7fdb40157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb3f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb400cd900 0x7fdb40157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39870/0 (socket says 172.21.15.79:39870) 2023-06-23T18:36:08.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb3f7fe700 1 -- 172.21.15.79:0/976419277 learned_addr learned my addr 172.21.15.79:0/976419277 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:08.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb3effd700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb400ce7f0 0x7fdb401e39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb3ffff700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb40159170 0x7fdb401e3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb3f7fe700 1 -- 172.21.15.79:0/976419277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb400ce7f0 msgr2=0x7fdb401e39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:08.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb3f7fe700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb400ce7f0 0x7fdb401e39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb3f7fe700 1 -- 172.21.15.79:0/976419277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb40159170 msgr2=0x7fdb401e3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:08.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb3f7fe700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb40159170 0x7fdb401e3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:08.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb3f7fe700 1 -- 172.21.15.79:0/976419277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb30009550 con 0x7fdb400cd900 2023-06-23T18:36:08.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb3ffff700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb40159170 0x7fdb401e3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:08.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb3f7fe700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb400cd900 0x7fdb40157be0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009130 tx=0x7fdb30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:08.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb37fff700 1 -- 172.21.15.79:0/976419277 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30003b50 con 0x7fdb400cd900 2023-06-23T18:36:08.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb37fff700 1 -- 172.21.15.79:0/976419277 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3001f070 con 0x7fdb400cd900 2023-06-23T18:36:08.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb37fff700 1 -- 172.21.15.79:0/976419277 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3000b660 con 0x7fdb400cd900 2023-06-23T18:36:08.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb45eb2700 1 -- 172.21.15.79:0/976419277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb400b3ba0 con 0x7fdb400cd900 2023-06-23T18:36:08.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.820+0000 7fdb45eb2700 1 -- 172.21.15.79:0/976419277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb401e4840 con 0x7fdb400cd900 2023-06-23T18:36:08.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.821+0000 7fdb45eb2700 1 -- 172.21.15.79:0/976419277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb400b4160 con 0x7fdb400cd900 2023-06-23T18:36:08.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.822+0000 7fdb37fff700 1 -- 172.21.15.79:0/976419277 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdb30021030 con 0x7fdb400cd900 2023-06-23T18:36:08.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.822+0000 7fdb37fff700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdb2006ace0 0x7fdb2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:08.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.822+0000 7fdb37fff700 1 -- 172.21.15.79:0/976419277 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdb3008a970 con 0x7fdb400cd900 2023-06-23T18:36:08.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.822+0000 7fdb3effd700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdb2006ace0 0x7fdb2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:08.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.823+0000 7fdb3effd700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdb2006ace0 0x7fdb2006d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb400c9cc0 tx=0x7fdb280090e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:08.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:08.826+0000 7fdb37fff700 1 -- 172.21.15.79:0/976419277 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdb30055ab0 con 0x7fdb400cd900 2023-06-23T18:36:09.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:09.059+0000 7fdb45eb2700 1 -- 172.21.15.79:0/976419277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdb400b4160 con 0x7fdb400cd900 2023-06-23T18:36:09.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:09.060+0000 7fdb37fff700 1 -- 172.21.15.79:0/976419277 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdb300590d0 con 0x7fdb400cd900 2023-06-23T18:36:09.065 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-23T18:36:09.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:09.061+0000 7fdb45eb2700 1 -- 172.21.15.79:0/976419277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdb2006ace0 msgr2=0x7fdb2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:09.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:09.061+0000 7fdb45eb2700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdb2006ace0 0x7fdb2006d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb400c9cc0 tx=0x7fdb280090e0 comp rx=0 tx=0).stop 2023-06-23T18:36:09.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:09.061+0000 7fdb45eb2700 1 -- 172.21.15.79:0/976419277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb400cd900 msgr2=0x7fdb40157be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:09.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:09.061+0000 7fdb45eb2700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb400cd900 0x7fdb40157be0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009130 tx=0x7fdb30004750 comp rx=0 tx=0).stop 2023-06-23T18:36:09.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:09.062+0000 7fdb45eb2700 1 -- 172.21.15.79:0/976419277 shutdown_connections 2023-06-23T18:36:09.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:09.062+0000 7fdb45eb2700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdb2006ace0 0x7fdb2006d190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:09.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:09.062+0000 7fdb45eb2700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb40159170 0x7fdb401e3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:09.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:09.062+0000 7fdb45eb2700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb400cd900 0x7fdb40157be0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:09.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:09.062+0000 7fdb45eb2700 1 --2- 172.21.15.79:0/976419277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb400ce7f0 0x7fdb401e39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:09.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:09.062+0000 7fdb45eb2700 1 -- 172.21.15.79:0/976419277 >> 172.21.15.79:0/976419277 conn(0x7fdb4006d660 msgr2=0x7fdb40159a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:09.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:09.062+0000 7fdb45eb2700 1 -- 172.21.15.79:0/976419277 shutdown_connections 2023-06-23T18:36:09.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:09.062+0000 7fdb45eb2700 1 -- 172.21.15.79:0/976419277 wait complete. 2023-06-23T18:36:09.078 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-23T18:36:09.079 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-23T18:36:09.079 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi079.front.sepia.ceph.com 2023-06-23T18:36:09.079 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-06-23T18:36:09.079 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-23T18:36:09.079 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-23T18:36:09.080 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-23T18:36:09.080 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-23T18:36:09.080 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-23T18:36:09.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:09.080 DEBUG:teuthology.orchestra.run.smithi079:> ip addr 2023-06-23T18:36:09.097 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:36:09.097 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:36:09.097 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:36:09.097 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:09.098 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2023-06-23T18:36:09.098 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:09.098 INFO:teuthology.orchestra.run.smithi079.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:36:09.098 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6a brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:09.098 INFO:teuthology.orchestra.run.smithi079.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:36:09.098 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6b brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:09.098 INFO:teuthology.orchestra.run.smithi079.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:36:09.099 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:86 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:09.099 INFO:teuthology.orchestra.run.smithi079.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-23T18:36:09.099 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:87 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:09.099 INFO:teuthology.orchestra.run.smithi079.stdout: inet 172.21.15.79/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-23T18:36:09.099 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft 42412sec preferred_lft 42412sec 2023-06-23T18:36:09.099 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::162a:fb46:60c1:412/64 scope link noprefixroute 2023-06-23T18:36:09.099 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:09.100 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-06-23T18:36:09.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:09.100 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-23T18:36:09.101 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-23T18:36:09.101 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add name ceph-brx type bridge 2023-06-23T18:36:09.101 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip addr flush dev ceph-brx 2023-06-23T18:36:09.101 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set ceph-brx up 2023-06-23T18:36:09.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-06-23T18:36:09.102 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-23T18:36:09.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:09.265 DEBUG:teuthology.orchestra.run.smithi079:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-23T18:36:09.293 INFO:teuthology.orchestra.run.smithi079.stdout:1 2023-06-23T18:36:09.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:09.295 DEBUG:teuthology.orchestra.run.smithi079:> route 2023-06-23T18:36:09.371 INFO:teuthology.orchestra.run.smithi079.stdout:Kernel IP routing table 2023-06-23T18:36:09.372 INFO:teuthology.orchestra.run.smithi079.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-23T18:36:09.372 INFO:teuthology.orchestra.run.smithi079.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-23T18:36:09.372 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-23T18:36:09.372 INFO:teuthology.orchestra.run.smithi079.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-23T18:36:09.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:09.373 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-23T18:36:09.373 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-23T18:36:09.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-23T18:36:09.374 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-23T18:36:09.374 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-23T18:36:09.374 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-23T18:36:09.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:09.625 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2023-06-23T18:36:09.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:09.642 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list-id 2023-06-23T18:36:09.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:09.698 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-23T18:36:09.698 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-23T18:36:09.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-23T18:36:09.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-06-23T18:36:09.699 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-23T18:36:09.791 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-06-23T18:36:09.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:09.792 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-23T18:36:09.792 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-23T18:36:09.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-06-23T18:36:09.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-06-23T18:36:09.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-06-23T18:36:09.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-06-23T18:36:09.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-06-23T18:36:09.794 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-23T18:36:10.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:10.017 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-23T18:36:10.018 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-23T18:36:10.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set brx.0 up 2023-06-23T18:36:10.019 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set dev brx.0 master ceph-brx 2023-06-23T18:36:10.019 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-23T18:36:10.134 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-06-23T18:36:10.134 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:36:10.134 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-06-23T18:36:10.156 INFO:teuthology.orchestra.run.smithi079.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-06-23T18:36:10.157 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:36:10.157 DEBUG:teuthology.orchestra.run.smithi079:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-06-23T18:36:10.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2023-06-23T18:36:10.279 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:36:10.280 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:36:10.345 INFO:teuthology.orchestra.run.smithi079.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:36:10.346 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:36:10.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:10.347 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2023-06-23T18:36:10.406 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-06-23T18:36:10.407 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-06-23T18:36:10.449 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2023-06-23T18:36:10.476 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:36:10.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:36:10.542 INFO:teuthology.orchestra.run.smithi079.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:36:10.542 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi079.stderr:2023-06-23T18:36:10.537+0000 7fb783d075c0 -1 init, newargv = 0x55e0ffc82ce0 newargc=15 2023-06-23T18:36:10.543 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi079.stderr:ceph-fuse[110131]: starting ceph client 2023-06-23T18:36:10.543 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:36:10.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:10.544 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2023-06-23T18:36:10.548 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi079.stderr:ceph-fuse[110131]: starting fuse 2023-06-23T18:36:10.563 INFO:teuthology.orchestra.run.smithi079.stdout:47 2023-06-23T18:36:10.564 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-06-23T18:36:10.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-23T18:36:10.564 DEBUG:teuthology.orchestra.run.smithi079:> import glob 2023-06-23T18:36:10.565 DEBUG:teuthology.orchestra.run.smithi079:> import re 2023-06-23T18:36:10.565 DEBUG:teuthology.orchestra.run.smithi079:> import os 2023-06-23T18:36:10.565 DEBUG:teuthology.orchestra.run.smithi079:> import subprocess 2023-06-23T18:36:10.565 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-23T18:36:10.566 DEBUG:teuthology.orchestra.run.smithi079:> def _find_admin_socket(client_name): 2023-06-23T18:36:10.566 DEBUG:teuthology.orchestra.run.smithi079:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-06-23T18:36:10.566 DEBUG:teuthology.orchestra.run.smithi079:> files = glob.glob(asok_path) 2023-06-23T18:36:10.566 DEBUG:teuthology.orchestra.run.smithi079:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-06-23T18:36:10.566 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-23T18:36:10.567 DEBUG:teuthology.orchestra.run.smithi079:> # Given a non-glob path, it better be there 2023-06-23T18:36:10.567 DEBUG:teuthology.orchestra.run.smithi079:> if "*" not in asok_path: 2023-06-23T18:36:10.567 DEBUG:teuthology.orchestra.run.smithi079:> assert(len(files) == 1) 2023-06-23T18:36:10.567 DEBUG:teuthology.orchestra.run.smithi079:> return files[0] 2023-06-23T18:36:10.568 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-23T18:36:10.568 DEBUG:teuthology.orchestra.run.smithi079:> for f in files: 2023-06-23T18:36:10.568 DEBUG:teuthology.orchestra.run.smithi079:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-23T18:36:10.568 DEBUG:teuthology.orchestra.run.smithi079:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-23T18:36:10.568 DEBUG:teuthology.orchestra.run.smithi079:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-23T18:36:10.569 DEBUG:teuthology.orchestra.run.smithi079:> contents = proc_f.read() 2023-06-23T18:36:10.569 DEBUG:teuthology.orchestra.run.smithi079:> if mountpoint in contents: 2023-06-23T18:36:10.569 DEBUG:teuthology.orchestra.run.smithi079:> return f 2023-06-23T18:36:10.569 DEBUG:teuthology.orchestra.run.smithi079:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-23T18:36:10.570 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-23T18:36:10.570 DEBUG:teuthology.orchestra.run.smithi079:> print(_find_admin_socket("client.0")) 2023-06-23T18:36:10.570 DEBUG:teuthology.orchestra.run.smithi079:> ' 2023-06-23T18:36:10.685 INFO:teuthology.orchestra.run.smithi079.stdout:/var/run/ceph/ceph-client.0.110131.asok 2023-06-23T18:36:10.868 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.110131.asok 2023-06-23T18:36:10.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:10.869 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.110131.asok status 2023-06-23T18:36:11.026 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-23T18:36:11.027 INFO:teuthology.orchestra.run.smithi079.stdout: "metadata": { 2023-06-23T18:36:11.027 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_sha1": "8627af3e0adcb765a3c249fcc209cba9f4873e1b", 2023-06-23T18:36:11.027 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:36:11.028 INFO:teuthology.orchestra.run.smithi079.stdout: "entity_id": "0", 2023-06-23T18:36:11.028 INFO:teuthology.orchestra.run.smithi079.stdout: "hostname": "smithi079", 2023-06-23T18:36:11.028 INFO:teuthology.orchestra.run.smithi079.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-06-23T18:36:11.028 INFO:teuthology.orchestra.run.smithi079.stdout: "pid": "110131", 2023-06-23T18:36:11.028 INFO:teuthology.orchestra.run.smithi079.stdout: "root": "/" 2023-06-23T18:36:11.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:11.029 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_count": 0, 2023-06-23T18:36:11.029 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_pinned_count": 0, 2023-06-23T18:36:11.029 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 4662, 2023-06-23T18:36:11.029 INFO:teuthology.orchestra.run.smithi079.stdout: "inst": { 2023-06-23T18:36:11.029 INFO:teuthology.orchestra.run.smithi079.stdout: "name": { 2023-06-23T18:36:11.030 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "client", 2023-06-23T18:36:11.030 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 4662 2023-06-23T18:36:11.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:11.030 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2023-06-23T18:36:11.030 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-06-23T18:36:11.030 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:36:11.030 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1296628217 2023-06-23T18:36:11.031 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:11.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:11.031 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2023-06-23T18:36:11.031 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-06-23T18:36:11.031 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:36:11.031 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1296628217 2023-06-23T18:36:11.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:11.032 INFO:teuthology.orchestra.run.smithi079.stdout: "inst_str": "client.4662 192.168.0.1:0/1296628217", 2023-06-23T18:36:11.032 INFO:teuthology.orchestra.run.smithi079.stdout: "addr_str": "192.168.0.1:0/1296628217", 2023-06-23T18:36:11.032 INFO:teuthology.orchestra.run.smithi079.stdout: "inode_count": 1, 2023-06-23T18:36:11.032 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_epoch": 8, 2023-06-23T18:36:11.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch": 20, 2023-06-23T18:36:11.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch_barrier": 0, 2023-06-23T18:36:11.033 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklisted": false, 2023-06-23T18:36:11.033 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs" 2023-06-23T18:36:11.034 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-23T18:36:11.036 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-23T18:36:11.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.152+0000 7f274838a700 1 Processor -- start 2023-06-23T18:36:11.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.152+0000 7f274838a700 1 -- start start 2023-06-23T18:36:11.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f274838a700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2740071220 0x7f27401608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:11.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f274838a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f274015c4e0 0x7f27400ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:11.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f274838a700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2740069e90 con 0x7f274015c110 2023-06-23T18:36:11.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f274838a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27400ccb30 con 0x7f274015c4e0 2023-06-23T18:36:11.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f274838a700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274006db00 con 0x7f2740071220 2023-06-23T18:36:11.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f2746927700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f274015c4e0 0x7f27400ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:11.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f2746927700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f274015c4e0 0x7f27400ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39892/0 (socket says 172.21.15.79:39892) 2023-06-23T18:36:11.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f2746927700 1 -- 172.21.15.79:0/1720956275 learned_addr learned my addr 172.21.15.79:0/1720956275 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:11.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f2745925700 1 --2- 172.21.15.79:0/1720956275 >> v2:172.21.15.155:3300/0 conn(0x7f2740071220 0x7f27401608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:11.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f2746927700 1 -- 172.21.15.79:0/1720956275 >> v2:172.21.15.155:3300/0 conn(0x7f2740071220 msgr2=0x7f27401608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:11.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f2746927700 1 --2- 172.21.15.79:0/1720956275 >> v2:172.21.15.155:3300/0 conn(0x7f2740071220 0x7f27401608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:11.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f2746927700 1 -- 172.21.15.79:0/1720956275 >> v1:172.21.15.155:6789/0 conn(0x7f274015c110 legacy=0x7f2740070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:11.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f2746927700 1 -- 172.21.15.79:0/1720956275 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27400ccd40 con 0x7f274015c4e0 2023-06-23T18:36:11.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f2746927700 1 --2- 172.21.15.79:0/1720956275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f274015c4e0 0x7f27400ce6c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2740070a10 tx=0x7f273800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e7dc4492c08392 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:11.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f2745124700 1 -- 172.21.15.79:0/1720956275 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273800e070 con 0x7f274015c4e0 2023-06-23T18:36:11.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f2745124700 1 -- 172.21.15.79:0/1720956275 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2738003a20 con 0x7f274015c4e0 2023-06-23T18:36:11.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.153+0000 7f2745124700 1 -- 172.21.15.79:0/1720956275 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738003dd0 con 0x7f274015c4e0 2023-06-23T18:36:11.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.154+0000 7f274838a700 1 -- 172.21.15.79:0/1720956275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f274015c4e0 msgr2=0x7f27400ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:11.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.154+0000 7f274838a700 1 --2- 172.21.15.79:0/1720956275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f274015c4e0 0x7f27400ce6c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2740070a10 tx=0x7f273800b330 comp rx=0 tx=0).stop 2023-06-23T18:36:11.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.154+0000 7f274838a700 1 -- 172.21.15.79:0/1720956275 shutdown_connections 2023-06-23T18:36:11.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.154+0000 7f274838a700 1 --2- 172.21.15.79:0/1720956275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f274015c4e0 0x7f27400ce6c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:11.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.154+0000 7f274838a700 1 --2- 172.21.15.79:0/1720956275 >> v2:172.21.15.155:3300/0 conn(0x7f2740071220 0x7f27401608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:11.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.154+0000 7f274838a700 1 -- 172.21.15.79:0/1720956275 >> 172.21.15.79:0/1720956275 conn(0x7f27400b17e0 msgr2=0x7f274006cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:11.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.154+0000 7f274838a700 1 -- 172.21.15.79:0/1720956275 shutdown_connections 2023-06-23T18:36:11.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.154+0000 7f274838a700 1 -- 172.21.15.79:0/1720956275 wait complete. 2023-06-23T18:36:11.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.155+0000 7f274838a700 1 Processor -- start 2023-06-23T18:36:11.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.155+0000 7f274838a700 1 -- start start 2023-06-23T18:36:11.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f274838a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2740071220 0x7f274015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:11.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f274838a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f274015c110 0x7f27400c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:11.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f274838a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f274015c4e0 0x7f27400abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:11.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f274838a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27400ae920 con 0x7f274015c110 2023-06-23T18:36:11.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f274838a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27400c5a00 con 0x7f274015c4e0 2023-06-23T18:36:11.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f274838a700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27400ae3a0 con 0x7f2740071220 2023-06-23T18:36:11.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f2745925700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f274015c110 0x7f27400c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:11.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f2745925700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f274015c110 0x7f27400c54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39896/0 (socket says 172.21.15.79:39896) 2023-06-23T18:36:11.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f2745925700 1 -- 172.21.15.79:0/3174942412 learned_addr learned my addr 172.21.15.79:0/3174942412 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:11.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f2745925700 1 -- 172.21.15.79:0/3174942412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2740071220 msgr2=0x7f274015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:11.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f2746927700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f274015c4e0 0x7f27400abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:11.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f2746126700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2740071220 0x7f274015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:11.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f2745925700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2740071220 0x7f274015a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:11.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f2745925700 1 -- 172.21.15.79:0/3174942412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f274015c4e0 msgr2=0x7f27400abc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:11.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f2745925700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f274015c4e0 0x7f27400abc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:11.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f2745925700 1 -- 172.21.15.79:0/3174942412 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2738009b30 con 0x7f274015c110 2023-06-23T18:36:11.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f2746927700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f274015c4e0 0x7f27400abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:11.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f2746126700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2740071220 0x7f274015a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:11.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.156+0000 7f2745925700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f274015c110 0x7f27400c54c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f273000c7f0 tx=0x7f273000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:11.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.157+0000 7f2736ffd700 1 -- 172.21.15.79:0/3174942412 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2730011800 con 0x7f274015c110 2023-06-23T18:36:11.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.157+0000 7f2736ffd700 1 -- 172.21.15.79:0/3174942412 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2730011960 con 0x7f274015c110 2023-06-23T18:36:11.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.157+0000 7f274838a700 1 -- 172.21.15.79:0/3174942412 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274015b3d0 con 0x7f274015c110 2023-06-23T18:36:11.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.157+0000 7f2736ffd700 1 -- 172.21.15.79:0/3174942412 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2730011cf0 con 0x7f274015c110 2023-06-23T18:36:11.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.157+0000 7f274838a700 1 -- 172.21.15.79:0/3174942412 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274015caf0 con 0x7f274015c110 2023-06-23T18:36:11.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.157+0000 7f274838a700 1 -- 172.21.15.79:0/3174942412 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274015c950 con 0x7f274015c110 2023-06-23T18:36:11.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.158+0000 7f2736ffd700 1 -- 172.21.15.79:0/3174942412 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2730018360 con 0x7f274015c110 2023-06-23T18:36:11.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.158+0000 7f2736ffd700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f272406ace0 0x7f272406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:11.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.158+0000 7f2736ffd700 1 -- 172.21.15.79:0/3174942412 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2730089500 con 0x7f274015c110 2023-06-23T18:36:11.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.158+0000 7f2746126700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f272406ace0 0x7f272406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:11.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.159+0000 7f2746126700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f272406ace0 0x7f272406d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f27400cc900 tx=0x7f273c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:11.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.163+0000 7f2736ffd700 1 -- 172.21.15.79:0/3174942412 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f274015c950 con 0x7f274015c110 2023-06-23T18:36:11.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.397+0000 7f274838a700 1 -- 172.21.15.79:0/3174942412 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f274015c950 con 0x7f274015c110 2023-06-23T18:36:11.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.397+0000 7f2736ffd700 1 -- 172.21.15.79:0/3174942412 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f274015c950 con 0x7f274015c110 2023-06-23T18:36:11.403 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-23T18:36:11.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.399+0000 7f274838a700 1 -- 172.21.15.79:0/3174942412 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f272406ace0 msgr2=0x7f272406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:11.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.399+0000 7f274838a700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f272406ace0 0x7f272406d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f27400cc900 tx=0x7f273c009040 comp rx=0 tx=0).stop 2023-06-23T18:36:11.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.399+0000 7f274838a700 1 -- 172.21.15.79:0/3174942412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f274015c110 msgr2=0x7f27400c54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:11.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.399+0000 7f274838a700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f274015c110 0x7f27400c54c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f273000c7f0 tx=0x7f273000cbb0 comp rx=0 tx=0).stop 2023-06-23T18:36:11.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.399+0000 7f274838a700 1 -- 172.21.15.79:0/3174942412 shutdown_connections 2023-06-23T18:36:11.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.399+0000 7f274838a700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f272406ace0 0x7f272406d190 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:11.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.399+0000 7f274838a700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f274015c4e0 0x7f27400abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:11.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.399+0000 7f274838a700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2740071220 0x7f274015a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:11.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.399+0000 7f274838a700 1 --2- 172.21.15.79:0/3174942412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f274015c110 0x7f27400c54c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:11.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.399+0000 7f274838a700 1 -- 172.21.15.79:0/3174942412 >> 172.21.15.79:0/3174942412 conn(0x7f27400b17e0 msgr2=0x7f27400b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:11.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.399+0000 7f274838a700 1 -- 172.21.15.79:0/3174942412 shutdown_connections 2023-06-23T18:36:11.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:11.399+0000 7f274838a700 1 -- 172.21.15.79:0/3174942412 wait complete. 2023-06-23T18:36:11.416 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-23T18:36:11.416 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-23T18:36:11.416 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi079.front.sepia.ceph.com 2023-06-23T18:36:11.417 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-06-23T18:36:11.417 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-06-23T18:36:11.417 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-23T18:36:11.417 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-23T18:36:11.417 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-23T18:36:11.418 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-06-23T18:36:11.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:11.418 DEBUG:teuthology.orchestra.run.smithi079:> ip addr 2023-06-23T18:36:11.435 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:36:11.435 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:36:11.435 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:36:11.435 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:11.436 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2023-06-23T18:36:11.436 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:11.436 INFO:teuthology.orchestra.run.smithi079.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:36:11.436 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6a brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:11.436 INFO:teuthology.orchestra.run.smithi079.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:36:11.436 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6b brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:11.436 INFO:teuthology.orchestra.run.smithi079.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:36:11.437 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:86 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:11.437 INFO:teuthology.orchestra.run.smithi079.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-23T18:36:11.437 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:87 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:11.437 INFO:teuthology.orchestra.run.smithi079.stdout: inet 172.21.15.79/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-23T18:36:11.437 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft 42409sec preferred_lft 42409sec 2023-06-23T18:36:11.437 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::162a:fb46:60c1:412/64 scope link noprefixroute 2023-06-23T18:36:11.437 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:11.438 INFO:teuthology.orchestra.run.smithi079.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:36:11.438 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 8a:d5:d7:85:fa:5d brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:11.438 INFO:teuthology.orchestra.run.smithi079.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-23T18:36:11.438 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:11.438 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::f86d:beff:fe28:e075/64 scope link 2023-06-23T18:36:11.438 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:11.438 INFO:teuthology.orchestra.run.smithi079.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-23T18:36:11.438 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 8a:d5:d7:85:fa:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-23T18:36:11.439 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::88d5:d7ff:fe85:fa5d/64 scope link tentative 2023-06-23T18:36:11.439 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:11.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:11.439 DEBUG:teuthology.orchestra.run.smithi079:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-23T18:36:11.500 INFO:teuthology.orchestra.run.smithi079.stdout:1 2023-06-23T18:36:11.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:11.502 DEBUG:teuthology.orchestra.run.smithi079:> route 2023-06-23T18:36:11.560 INFO:teuthology.orchestra.run.smithi079.stdout:Kernel IP routing table 2023-06-23T18:36:11.560 INFO:teuthology.orchestra.run.smithi079.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-23T18:36:11.561 INFO:teuthology.orchestra.run.smithi079.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-23T18:36:11.561 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-23T18:36:11.561 INFO:teuthology.orchestra.run.smithi079.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-23T18:36:11.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:11.561 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-23T18:36:11.562 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-23T18:36:11.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-23T18:36:11.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-23T18:36:11.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-23T18:36:11.563 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-23T18:36:11.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:11.665 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2023-06-23T18:36:11.681 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-06-23T18:36:11.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:11.682 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list-id 2023-06-23T18:36:11.738 INFO:teuthology.orchestra.run.smithi079.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-06-23T18:36:11.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:11.739 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-23T18:36:11.739 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-23T18:36:11.739 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-06-23T18:36:11.740 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-06-23T18:36:11.740 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-23T18:36:11.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:11.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-06-23T18:36:11.866 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:36:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:36:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:36:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2023-06-23T18:36:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:11.868 INFO:teuthology.orchestra.run.smithi079.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:36:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 46:2e:10:2e:db:6f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-23T18:36:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-23T18:36:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::442e:10ff:fe2e:db6f/64 scope link 2023-06-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:11.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:11.880 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-06-23T18:36:11.957 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:36:11.958 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:36:11.958 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:36:11.958 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:11.958 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2023-06-23T18:36:11.958 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:11.958 INFO:teuthology.orchestra.run.smithi079.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:36:11.959 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 46:2e:10:2e:db:6f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-23T18:36:11.959 INFO:teuthology.orchestra.run.smithi079.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-23T18:36:11.959 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:11.959 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::442e:10ff:fe2e:db6f/64 scope link 2023-06-23T18:36:11.959 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:11.969 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-06-23T18:36:11.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:11.971 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-23T18:36:11.971 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-23T18:36:11.972 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-06-23T18:36:11.972 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-06-23T18:36:11.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-06-23T18:36:11.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-06-23T18:36:11.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-06-23T18:36:11.974 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-23T18:36:12.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:12.217 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-23T18:36:12.217 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-23T18:36:12.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set brx.1 up 2023-06-23T18:36:12.218 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set dev brx.1 master ceph-brx 2023-06-23T18:36:12.218 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-23T18:36:12.327 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-06-23T18:36:12.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:36:12.328 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-06-23T18:36:12.350 INFO:teuthology.orchestra.run.smithi079.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-06-23T18:36:12.351 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:36:12.351 DEBUG:teuthology.orchestra.run.smithi079:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-06-23T18:36:12.406 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2023-06-23T18:36:12.473 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:36:12.473 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:36:12.539 INFO:teuthology.orchestra.run.smithi079.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:36:12.540 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:36:12.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:12.541 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2023-06-23T18:36:12.597 INFO:teuthology.orchestra.run.smithi079.stdout:47 2023-06-23T18:36:12.597 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2023-06-23T18:36:12.598 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-06-23T18:36:12.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2023-06-23T18:36:12.665 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:36:12.666 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:36:12.727 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi079.stderr:2023-06-23T18:36:12.723+0000 7f38dc18c5c0 -1 init, newargv = 0x5586d49ed9d0 newargc=15 2023-06-23T18:36:12.728 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi079.stderr:ceph-fuse[110779]: starting ceph client 2023-06-23T18:36:12.732 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi079.stderr:ceph-fuse[110779]: starting fuse 2023-06-23T18:36:12.735 INFO:teuthology.orchestra.run.smithi079.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:36:12.735 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:36:12.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:12.737 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2023-06-23T18:36:12.793 INFO:teuthology.orchestra.run.smithi079.stdout:47 2023-06-23T18:36:12.793 INFO:teuthology.orchestra.run.smithi079.stdout:48 2023-06-23T18:36:12.793 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2023-06-23T18:36:12.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-23T18:36:12.794 DEBUG:teuthology.orchestra.run.smithi079:> import glob 2023-06-23T18:36:12.794 DEBUG:teuthology.orchestra.run.smithi079:> import re 2023-06-23T18:36:12.794 DEBUG:teuthology.orchestra.run.smithi079:> import os 2023-06-23T18:36:12.795 DEBUG:teuthology.orchestra.run.smithi079:> import subprocess 2023-06-23T18:36:12.795 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-23T18:36:12.795 DEBUG:teuthology.orchestra.run.smithi079:> def _find_admin_socket(client_name): 2023-06-23T18:36:12.796 DEBUG:teuthology.orchestra.run.smithi079:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-06-23T18:36:12.797 DEBUG:teuthology.orchestra.run.smithi079:> files = glob.glob(asok_path) 2023-06-23T18:36:12.797 DEBUG:teuthology.orchestra.run.smithi079:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-06-23T18:36:12.798 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-23T18:36:12.798 DEBUG:teuthology.orchestra.run.smithi079:> # Given a non-glob path, it better be there 2023-06-23T18:36:12.798 DEBUG:teuthology.orchestra.run.smithi079:> if "*" not in asok_path: 2023-06-23T18:36:12.798 DEBUG:teuthology.orchestra.run.smithi079:> assert(len(files) == 1) 2023-06-23T18:36:12.798 DEBUG:teuthology.orchestra.run.smithi079:> return files[0] 2023-06-23T18:36:12.799 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-23T18:36:12.799 DEBUG:teuthology.orchestra.run.smithi079:> for f in files: 2023-06-23T18:36:12.799 DEBUG:teuthology.orchestra.run.smithi079:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-23T18:36:12.799 DEBUG:teuthology.orchestra.run.smithi079:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-23T18:36:12.800 DEBUG:teuthology.orchestra.run.smithi079:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-23T18:36:12.800 DEBUG:teuthology.orchestra.run.smithi079:> contents = proc_f.read() 2023-06-23T18:36:12.800 DEBUG:teuthology.orchestra.run.smithi079:> if mountpoint in contents: 2023-06-23T18:36:12.800 DEBUG:teuthology.orchestra.run.smithi079:> return f 2023-06-23T18:36:12.801 DEBUG:teuthology.orchestra.run.smithi079:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-23T18:36:12.801 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-23T18:36:12.801 DEBUG:teuthology.orchestra.run.smithi079:> print(_find_admin_socket("client.2")) 2023-06-23T18:36:12.801 DEBUG:teuthology.orchestra.run.smithi079:> ' 2023-06-23T18:36:12.914 INFO:teuthology.orchestra.run.smithi079.stdout:/var/run/ceph/ceph-client.2.110779.asok 2023-06-23T18:36:13.097 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.110779.asok 2023-06-23T18:36:13.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:13.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.110779.asok status 2023-06-23T18:36:13.253 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-23T18:36:13.253 INFO:teuthology.orchestra.run.smithi079.stdout: "metadata": { 2023-06-23T18:36:13.253 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_sha1": "8627af3e0adcb765a3c249fcc209cba9f4873e1b", 2023-06-23T18:36:13.253 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:36:13.253 INFO:teuthology.orchestra.run.smithi079.stdout: "entity_id": "2", 2023-06-23T18:36:13.254 INFO:teuthology.orchestra.run.smithi079.stdout: "hostname": "smithi079", 2023-06-23T18:36:13.254 INFO:teuthology.orchestra.run.smithi079.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-06-23T18:36:13.254 INFO:teuthology.orchestra.run.smithi079.stdout: "pid": "110779", 2023-06-23T18:36:13.254 INFO:teuthology.orchestra.run.smithi079.stdout: "root": "/" 2023-06-23T18:36:13.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:13.254 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_count": 0, 2023-06-23T18:36:13.254 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_pinned_count": 0, 2023-06-23T18:36:13.254 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 4487, 2023-06-23T18:36:13.255 INFO:teuthology.orchestra.run.smithi079.stdout: "inst": { 2023-06-23T18:36:13.255 INFO:teuthology.orchestra.run.smithi079.stdout: "name": { 2023-06-23T18:36:13.255 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "client", 2023-06-23T18:36:13.255 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 4487 2023-06-23T18:36:13.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2023-06-23T18:36:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-06-23T18:36:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:0", 2023-06-23T18:36:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2770351625 2023-06-23T18:36:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2023-06-23T18:36:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-06-23T18:36:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:0", 2023-06-23T18:36:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2770351625 2023-06-23T18:36:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "inst_str": "client.4487 172.21.15.79:0/2770351625", 2023-06-23T18:36:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "addr_str": "172.21.15.79:0/2770351625", 2023-06-23T18:36:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "inode_count": 1, 2023-06-23T18:36:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_epoch": 8, 2023-06-23T18:36:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch": 20, 2023-06-23T18:36:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch_barrier": 0, 2023-06-23T18:36:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklisted": false, 2023-06-23T18:36:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs" 2023-06-23T18:36:13.258 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-23T18:36:13.262 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-23T18:36:13.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.377+0000 7f9891081700 1 Processor -- start 2023-06-23T18:36:13.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.377+0000 7f9891081700 1 -- start start 2023-06-23T18:36:13.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.378+0000 7f9891081700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f988c071490 0x7f988c071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:13.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.378+0000 7f9891081700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988c1608f0 con 0x7f988c0aee30 2023-06-23T18:36:13.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.378+0000 7f9891081700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988c160e20 con 0x7f988c0b1270 2023-06-23T18:36:13.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.378+0000 7f9891081700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988c06acb0 con 0x7f988c071490 2023-06-23T18:36:13.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.378+0000 7f988ad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f988c071490 0x7f988c071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:13.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.378+0000 7f988b59e700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f988c0aee30 0x7f988c06a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42968/0 (socket says 172.21.15.79:42968) 2023-06-23T18:36:13.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.378+0000 7f988b59e700 1 -- 172.21.15.79:0/2462933041 learned_addr learned my addr 172.21.15.79:0/2462933041 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:13.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.378+0000 7f988ad9d700 1 -- 172.21.15.79:0/2462933041 >> v1:172.21.15.155:6789/0 conn(0x7f988c0b1270 legacy=0x7f988c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:13.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.378+0000 7f988ad9d700 1 -- 172.21.15.79:0/2462933041 >> v1:172.21.15.79:6789/0 conn(0x7f988c0aee30 legacy=0x7f988c06a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:13.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.378+0000 7f9889d9b700 1 -- 172.21.15.79:0/2462933041 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360538162 0 0) 0x7f988c1608f0 con 0x7f988c0aee30 2023-06-23T18:36:13.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.378+0000 7f988ad9d700 1 -- 172.21.15.79:0/2462933041 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c0cd4f0 con 0x7f988c071490 2023-06-23T18:36:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.378+0000 7f988ad9d700 1 --2- 172.21.15.79:0/2462933041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c071490 0x7f988c071860 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9880009840 tx=0x7f9880009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5935ad944190b59 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.378+0000 7f9889d9b700 1 -- 172.21.15.79:0/2462933041 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880004230 con 0x7f988c071490 2023-06-23T18:36:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.379+0000 7f9889d9b700 1 -- 172.21.15.79:0/2462933041 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9880004390 con 0x7f988c071490 2023-06-23T18:36:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.379+0000 7f9889d9b700 1 -- 172.21.15.79:0/2462933041 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98800036a0 con 0x7f988c071490 2023-06-23T18:36:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.379+0000 7f9891081700 1 -- 172.21.15.79:0/2462933041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c071490 msgr2=0x7f988c071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.379+0000 7f9891081700 1 --2- 172.21.15.79:0/2462933041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c071490 0x7f988c071860 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9880009840 tx=0x7f9880009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.379+0000 7f9891081700 1 -- 172.21.15.79:0/2462933041 shutdown_connections 2023-06-23T18:36:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.379+0000 7f9891081700 1 --2- 172.21.15.79:0/2462933041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c071490 0x7f988c071860 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:13.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.379+0000 7f9891081700 1 -- 172.21.15.79:0/2462933041 >> 172.21.15.79:0/2462933041 conn(0x7f988c06d660 msgr2=0x7f988c06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:13.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.379+0000 7f9891081700 1 -- 172.21.15.79:0/2462933041 shutdown_connections 2023-06-23T18:36:13.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.379+0000 7f9891081700 1 -- 172.21.15.79:0/2462933041 wait complete. 2023-06-23T18:36:13.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.380+0000 7f9891081700 1 Processor -- start 2023-06-23T18:36:13.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.381+0000 7f9891081700 1 -- start start 2023-06-23T18:36:13.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.381+0000 7f9891081700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f988c071490 0x7f988c0b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:13.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.381+0000 7f9891081700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c0aee30 0x7f988c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:13.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.381+0000 7f9891081700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f988c0b1270 0x7f988c0b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:13.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.381+0000 7f9891081700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0b2e50 con 0x7f988c0aee30 2023-06-23T18:36:13.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.381+0000 7f9891081700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0c9840 con 0x7f988c0b1270 2023-06-23T18:36:13.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.381+0000 7f9891081700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0c9560 con 0x7f988c071490 2023-06-23T18:36:13.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.381+0000 7f988a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c0aee30 0x7f988c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:13.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.381+0000 7f988a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c0aee30 0x7f988c0b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39930/0 (socket says 172.21.15.79:39930) 2023-06-23T18:36:13.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.381+0000 7f988a59c700 1 -- 172.21.15.79:0/4176330431 learned_addr learned my addr 172.21.15.79:0/4176330431 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:13.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.381+0000 7f988b59e700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f988c0b1270 0x7f988c0b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:13.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.381+0000 7f988ad9d700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f988c071490 0x7f988c0b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:13.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.382+0000 7f988a59c700 1 -- 172.21.15.79:0/4176330431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f988c071490 msgr2=0x7f988c0b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:13.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.382+0000 7f988a59c700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f988c071490 0x7f988c0b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:13.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.382+0000 7f988a59c700 1 -- 172.21.15.79:0/4176330431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f988c0b1270 msgr2=0x7f988c0b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:13.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.382+0000 7f988a59c700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f988c0b1270 0x7f988c0b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:13.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.382+0000 7f988a59c700 1 -- 172.21.15.79:0/4176330431 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98800094a0 con 0x7f988c0aee30 2023-06-23T18:36:13.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.382+0000 7f988ad9d700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f988c071490 0x7f988c0b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:13.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.382+0000 7f988a59c700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c0aee30 0x7f988c0b3820 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f987c00b450 tx=0x7f987c00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:13.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.382+0000 7f98737fe700 1 -- 172.21.15.79:0/4176330431 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c004020 con 0x7f988c0aee30 2023-06-23T18:36:13.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.382+0000 7f98737fe700 1 -- 172.21.15.79:0/4176330431 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c004180 con 0x7f988c0aee30 2023-06-23T18:36:13.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.382+0000 7f98737fe700 1 -- 172.21.15.79:0/4176330431 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c004510 con 0x7f988c0aee30 2023-06-23T18:36:13.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.382+0000 7f9891081700 1 -- 172.21.15.79:0/4176330431 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988c0b2800 con 0x7f988c0aee30 2023-06-23T18:36:13.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.382+0000 7f9891081700 1 -- 172.21.15.79:0/4176330431 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c156be0 con 0x7f988c0aee30 2023-06-23T18:36:13.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.383+0000 7f9891081700 1 -- 172.21.15.79:0/4176330431 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988c0b3000 con 0x7f988c0aee30 2023-06-23T18:36:13.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.383+0000 7f98737fe700 1 -- 172.21.15.79:0/4176330431 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f987c003680 con 0x7f988c0aee30 2023-06-23T18:36:13.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.384+0000 7f98737fe700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f987806ace0 0x7f987806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:13.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.384+0000 7f98737fe700 1 -- 172.21.15.79:0/4176330431 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f987c089030 con 0x7f988c0aee30 2023-06-23T18:36:13.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.384+0000 7f988ad9d700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f987806ace0 0x7f987806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:13.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.384+0000 7f988ad9d700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f987806ace0 0x7f987806d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f98800090b0 tx=0x7f9880008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:13.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.388+0000 7f98737fe700 1 -- 172.21.15.79:0/4176330431 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f988c0b3000 con 0x7f988c0aee30 2023-06-23T18:36:13.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.613+0000 7f9891081700 1 -- 172.21.15.79:0/4176330431 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f988c0b3000 con 0x7f988c0aee30 2023-06-23T18:36:13.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.613+0000 7f98737fe700 1 -- 172.21.15.79:0/4176330431 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f988c0b3000 con 0x7f988c0aee30 2023-06-23T18:36:13.619 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-23T18:36:13.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.615+0000 7f9891081700 1 -- 172.21.15.79:0/4176330431 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f987806ace0 msgr2=0x7f987806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:13.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.615+0000 7f9891081700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f987806ace0 0x7f987806d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f98800090b0 tx=0x7f9880008030 comp rx=0 tx=0).stop 2023-06-23T18:36:13.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.615+0000 7f9891081700 1 -- 172.21.15.79:0/4176330431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c0aee30 msgr2=0x7f988c0b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:13.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.615+0000 7f9891081700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c0aee30 0x7f988c0b3820 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f987c00b450 tx=0x7f987c00b760 comp rx=0 tx=0).stop 2023-06-23T18:36:13.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.615+0000 7f9891081700 1 -- 172.21.15.79:0/4176330431 shutdown_connections 2023-06-23T18:36:13.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.615+0000 7f9891081700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f987806ace0 0x7f987806d190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:13.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.615+0000 7f9891081700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f988c0b1270 0x7f988c0b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:13.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.615+0000 7f9891081700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f988c071490 0x7f988c0b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:13.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.615+0000 7f9891081700 1 --2- 172.21.15.79:0/4176330431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c0aee30 0x7f988c0b3820 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:13.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.615+0000 7f9891081700 1 -- 172.21.15.79:0/4176330431 >> 172.21.15.79:0/4176330431 conn(0x7f988c06d660 msgr2=0x7f988c0b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:13.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.616+0000 7f9891081700 1 -- 172.21.15.79:0/4176330431 shutdown_connections 2023-06-23T18:36:13.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:13.616+0000 7f9891081700 1 -- 172.21.15.79:0/4176330431 wait complete. 2023-06-23T18:36:13.632 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-23T18:36:13.632 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-23T18:36:13.633 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-06-23T18:36:13.633 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-06-23T18:36:13.633 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-06-23T18:36:13.633 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-23T18:36:13.633 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-23T18:36:13.634 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-23T18:36:13.634 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-06-23T18:36:13.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:13.634 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-06-23T18:36:13.656 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:36:13.656 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:36:13.657 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:36:13.657 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:13.657 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-06-23T18:36:13.657 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:13.657 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:36:13.658 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:13.658 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:36:13.658 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:13.658 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:36:13.659 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:13.659 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-23T18:36:13.659 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:13.659 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-23T18:36:13.659 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42414sec preferred_lft 42414sec 2023-06-23T18:36:13.660 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::1cf2:57d9:5266:4005/64 scope link noprefixroute 2023-06-23T18:36:13.660 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:13.660 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-06-23T18:36:13.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:13.661 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-06-23T18:36:13.662 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-06-23T18:36:13.662 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add name ceph-brx type bridge 2023-06-23T18:36:13.662 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip addr flush dev ceph-brx 2023-06-23T18:36:13.663 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set ceph-brx up 2023-06-23T18:36:13.663 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-06-23T18:36:13.663 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-06-23T18:36:13.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:13.841 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-23T18:36:13.869 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-06-23T18:36:13.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:13.871 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-06-23T18:36:13.942 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-06-23T18:36:13.942 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-23T18:36:13.942 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-23T18:36:13.943 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-23T18:36:13.943 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-23T18:36:13.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:13.943 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-06-23T18:36:13.944 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-06-23T18:36:13.944 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-23T18:36:13.944 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-23T18:36:13.944 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-23T18:36:13.945 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-06-23T18:36:14.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:14.213 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-06-23T18:36:14.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:14.231 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list-id 2023-06-23T18:36:14.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:14.288 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-06-23T18:36:14.288 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-06-23T18:36:14.288 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-06-23T18:36:14.288 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-06-23T18:36:14.289 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-06-23T18:36:14.381 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-06-23T18:36:14.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:14.382 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-06-23T18:36:14.382 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-06-23T18:36:14.383 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-06-23T18:36:14.383 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-06-23T18:36:14.384 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-06-23T18:36:14.384 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-06-23T18:36:14.384 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-06-23T18:36:14.384 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-06-23T18:36:14.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:14.576 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-06-23T18:36:14.576 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-06-23T18:36:14.577 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2023-06-23T18:36:14.577 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set dev brx.0 master ceph-brx 2023-06-23T18:36:14.577 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-06-23T18:36:14.676 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-06-23T18:36:14.676 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:36:14.676 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-06-23T18:36:14.696 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-06-23T18:36:14.697 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:36:14.697 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-06-23T18:36:14.753 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-06-23T18:36:14.821 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:36:14.821 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:36:14.887 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:36:14.887 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:36:14.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:14.888 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-06-23T18:36:14.945 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-06-23T18:36:14.945 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-06-23T18:36:14.990 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-06-23T18:36:15.016 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:36:15.016 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:36:15.082 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:36:15.083 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:36:15.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:15.084 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-06-23T18:36:15.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2023-06-23T18:36:15.084+0000 7fcc742385c0 -1 init, newargv = 0x55a0d6b8bb00 newargc=15 2023-06-23T18:36:15.089 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[107382]: starting ceph client 2023-06-23T18:36:15.093 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[107382]: starting fuse 2023-06-23T18:36:15.105 INFO:teuthology.orchestra.run.smithi155.stdout:46 2023-06-23T18:36:15.105 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-06-23T18:36:15.105 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-23T18:36:15.106 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-06-23T18:36:15.106 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-06-23T18:36:15.106 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-06-23T18:36:15.106 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-06-23T18:36:15.107 DEBUG:teuthology.orchestra.run.smithi155:> 2023-06-23T18:36:15.107 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-06-23T18:36:15.107 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-06-23T18:36:15.107 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-06-23T18:36:15.108 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-06-23T18:36:15.108 DEBUG:teuthology.orchestra.run.smithi155:> 2023-06-23T18:36:15.108 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-06-23T18:36:15.108 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-06-23T18:36:15.108 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-06-23T18:36:15.109 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-06-23T18:36:15.109 DEBUG:teuthology.orchestra.run.smithi155:> 2023-06-23T18:36:15.109 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-06-23T18:36:15.109 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-23T18:36:15.110 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-23T18:36:15.110 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-23T18:36:15.110 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-06-23T18:36:15.110 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-06-23T18:36:15.110 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-06-23T18:36:15.111 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-23T18:36:15.111 DEBUG:teuthology.orchestra.run.smithi155:> 2023-06-23T18:36:15.111 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2023-06-23T18:36:15.111 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-06-23T18:36:15.228 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.107382.asok 2023-06-23T18:36:15.410 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.107382.asok 2023-06-23T18:36:15.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:15.411 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.107382.asok status 2023-06-23T18:36:15.566 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-06-23T18:36:15.566 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-06-23T18:36:15.566 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "8627af3e0adcb765a3c249fcc209cba9f4873e1b", 2023-06-23T18:36:15.566 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:36:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2023-06-23T18:36:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-06-23T18:36:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-06-23T18:36:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "107382", 2023-06-23T18:36:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-06-23T18:36:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-06-23T18:36:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-06-23T18:36:15.568 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-06-23T18:36:15.568 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 4496, 2023-06-23T18:36:15.568 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-06-23T18:36:15.568 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-06-23T18:36:15.568 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-06-23T18:36:15.569 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 4496 2023-06-23T18:36:15.569 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-06-23T18:36:15.569 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-06-23T18:36:15.569 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-06-23T18:36:15.569 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:36:15.569 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 940317374 2023-06-23T18:36:15.569 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-06-23T18:36:15.570 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-06-23T18:36:15.570 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-06-23T18:36:15.570 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-06-23T18:36:15.570 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:36:15.570 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 940317374 2023-06-23T18:36:15.570 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-06-23T18:36:15.570 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.4496 192.168.0.1:0/940317374", 2023-06-23T18:36:15.571 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/940317374", 2023-06-23T18:36:15.571 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-06-23T18:36:15.571 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 8, 2023-06-23T18:36:15.571 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 20, 2023-06-23T18:36:15.571 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-06-23T18:36:15.571 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-06-23T18:36:15.571 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-06-23T18:36:15.571 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-06-23T18:36:15.577 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-23T18:36:15.709 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.705+0000 7f0afea5e700 1 Processor -- start 2023-06-23T18:36:15.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.706+0000 7f0afea5e700 1 -- start start 2023-06-23T18:36:15.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.706+0000 7f0afea5e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0af80b83c0 0x7f0af80cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:15.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.706+0000 7f0afea5e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0af80d3710 0x7f0af80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:15.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.706+0000 7f0afea5e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af80cfee0 con 0x7f0af80cfa90 2023-06-23T18:36:15.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.706+0000 7f0afea5e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af80d0a80 con 0x7f0af80d3710 2023-06-23T18:36:15.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.706+0000 7f0afea5e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80b8790 con 0x7f0af80b83c0 2023-06-23T18:36:15.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.706+0000 7f0af77fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0af80d3710 0x7f0af80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:15.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.706+0000 7f0af77fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0af80d3710 0x7f0af80cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39938/0 (socket says 172.21.15.79:39938) 2023-06-23T18:36:15.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.706+0000 7f0af77fe700 1 -- 172.21.15.79:0/4231298871 learned_addr learned my addr 172.21.15.79:0/4231298871 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:15.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.706+0000 7f0afcffb700 1 --1- 172.21.15.79:0/4231298871 >> v1:172.21.15.79:6789/0 conn(0x7f0af80cfa90 0x7f0af815feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:42980/0 (socket says 172.21.15.79:42980) 2023-06-23T18:36:15.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.707+0000 7f0af7fff700 1 --2- 172.21.15.79:0/4231298871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0af80b83c0 0x7f0af80cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:15.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.707+0000 7f0af77fe700 1 -- 172.21.15.79:0/4231298871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0af80b83c0 msgr2=0x7f0af80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:15.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.707+0000 7f0af77fe700 1 --2- 172.21.15.79:0/4231298871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0af80b83c0 0x7f0af80cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:15.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.707+0000 7f0af77fe700 1 -- 172.21.15.79:0/4231298871 >> v1:172.21.15.79:6789/0 conn(0x7f0af80cfa90 legacy=0x7f0af815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:15.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.707+0000 7f0af77fe700 1 -- 172.21.15.79:0/4231298871 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80b4e70 con 0x7f0af80d3710 2023-06-23T18:36:15.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.707+0000 7f0af6ffd700 1 -- 172.21.15.79:0/4231298871 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871832329 0 0) 0x7f0af0001490 con 0x7f0af80cfa90 2023-06-23T18:36:15.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.707+0000 7f0af77fe700 1 --2- 172.21.15.79:0/4231298871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0af80d3710 0x7f0af80cf550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80098f0 tx=0x7f0ae8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d495542fdd44e0f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:15.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.707+0000 7f0af6ffd700 1 -- 172.21.15.79:0/4231298871 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8011830 con 0x7f0af80d3710 2023-06-23T18:36:15.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.707+0000 7f0af6ffd700 1 -- 172.21.15.79:0/4231298871 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae8011990 con 0x7f0af80d3710 2023-06-23T18:36:15.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.707+0000 7f0af6ffd700 1 -- 172.21.15.79:0/4231298871 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8011d20 con 0x7f0af80d3710 2023-06-23T18:36:15.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.708+0000 7f0afea5e700 1 -- 172.21.15.79:0/4231298871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0af80d3710 msgr2=0x7f0af80cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:15.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.708+0000 7f0afea5e700 1 --2- 172.21.15.79:0/4231298871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0af80d3710 0x7f0af80cf550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80098f0 tx=0x7f0ae8009c00 comp rx=0 tx=0).stop 2023-06-23T18:36:15.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.708+0000 7f0afea5e700 1 -- 172.21.15.79:0/4231298871 shutdown_connections 2023-06-23T18:36:15.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.708+0000 7f0afea5e700 1 --2- 172.21.15.79:0/4231298871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0af80b83c0 0x7f0af80cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:15.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.708+0000 7f0afea5e700 1 --2- 172.21.15.79:0/4231298871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0af80d3710 0x7f0af80cf550 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:15.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.708+0000 7f0afea5e700 1 -- 172.21.15.79:0/4231298871 >> 172.21.15.79:0/4231298871 conn(0x7f0af806b350 msgr2=0x7f0af806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:15.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.708+0000 7f0afea5e700 1 -- 172.21.15.79:0/4231298871 shutdown_connections 2023-06-23T18:36:15.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.708+0000 7f0afea5e700 1 -- 172.21.15.79:0/4231298871 wait complete. 2023-06-23T18:36:15.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.709+0000 7f0afea5e700 1 Processor -- start 2023-06-23T18:36:15.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.709+0000 7f0afea5e700 1 -- start start 2023-06-23T18:36:15.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0afea5e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0af80b83c0 0x7f0af8150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:15.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0afea5e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0af80cfa90 0x7f0af80c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:15.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0afea5e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0af80d3710 0x7f0af80b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:15.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0afea5e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80c8730 con 0x7f0af80b83c0 2023-06-23T18:36:15.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0afea5e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af815c930 con 0x7f0af80cfa90 2023-06-23T18:36:15.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0afea5e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80c8450 con 0x7f0af80d3710 2023-06-23T18:36:15.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0af7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0af80b83c0 0x7f0af8150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:15.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0af7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0af80b83c0 0x7f0af8150360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39942/0 (socket says 172.21.15.79:39942) 2023-06-23T18:36:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0af7fff700 1 -- 172.21.15.79:0/3657194821 learned_addr learned my addr 172.21.15.79:0/3657194821 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0afcffb700 1 --2- 172.21.15.79:0/3657194821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0af80d3710 0x7f0af80b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0af7fff700 1 -- 172.21.15.79:0/3657194821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0af80d3710 msgr2=0x7f0af80b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0af7fff700 1 --2- 172.21.15.79:0/3657194821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0af80d3710 0x7f0af80b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:15.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0af7fff700 1 -- 172.21.15.79:0/3657194821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0af80cfa90 msgr2=0x7f0af80c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:15.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0af7fff700 1 --2- 172.21.15.79:0/3657194821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0af80cfa90 0x7f0af80c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:15.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0af7fff700 1 -- 172.21.15.79:0/3657194821 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae8009550 con 0x7f0af80b83c0 2023-06-23T18:36:15.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0af7fff700 1 --2- 172.21.15.79:0/3657194821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0af80b83c0 0x7f0af8150360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000e870 tx=0x7f0ae000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:15.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0af4ff9700 1 -- 172.21.15.79:0/3657194821 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0009590 con 0x7f0af80b83c0 2023-06-23T18:36:15.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.710+0000 7f0af4ff9700 1 -- 172.21.15.79:0/3657194821 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae000ee70 con 0x7f0af80b83c0 2023-06-23T18:36:15.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.711+0000 7f0afea5e700 1 -- 172.21.15.79:0/3657194821 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af815cb80 con 0x7f0af80b83c0 2023-06-23T18:36:15.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.711+0000 7f0af4ff9700 1 -- 172.21.15.79:0/3657194821 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae000ca50 con 0x7f0af80b83c0 2023-06-23T18:36:15.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.711+0000 7f0afea5e700 1 -- 172.21.15.79:0/3657194821 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af815d140 con 0x7f0af80b83c0 2023-06-23T18:36:15.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.711+0000 7f0afea5e700 1 -- 172.21.15.79:0/3657194821 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af80c7930 con 0x7f0af80b83c0 2023-06-23T18:36:15.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.712+0000 7f0af4ff9700 1 -- 172.21.15.79:0/3657194821 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0ae000cbb0 con 0x7f0af80b83c0 2023-06-23T18:36:15.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.712+0000 7f0af4ff9700 1 --2- 172.21.15.79:0/3657194821 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0ad806ace0 0x7f0ad806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:15.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.712+0000 7f0af4ff9700 1 -- 172.21.15.79:0/3657194821 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0ae0089450 con 0x7f0af80b83c0 2023-06-23T18:36:15.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.712+0000 7f0af77fe700 1 --2- 172.21.15.79:0/3657194821 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0ad806ace0 0x7f0ad806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:15.725 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.713+0000 7f0af77fe700 1 --2- 172.21.15.79:0/3657194821 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0ad806ace0 0x7f0ad806d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0af80d3b20 tx=0x7f0ae8010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:15.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.717+0000 7f0af4ff9700 1 -- 172.21.15.79:0/3657194821 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0ae0057f90 con 0x7f0af80b83c0 2023-06-23T18:36:15.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.948+0000 7f0afea5e700 1 -- 172.21.15.79:0/3657194821 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0af815f7d0 con 0x7f0af80b83c0 2023-06-23T18:36:15.953 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.949+0000 7f0af4ff9700 1 -- 172.21.15.79:0/3657194821 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0ae000ce60 con 0x7f0af80b83c0 2023-06-23T18:36:15.953 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-23T18:36:15.955 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.951+0000 7f0afea5e700 1 -- 172.21.15.79:0/3657194821 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0ad806ace0 msgr2=0x7f0ad806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:15.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.951+0000 7f0afea5e700 1 --2- 172.21.15.79:0/3657194821 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0ad806ace0 0x7f0ad806d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0af80d3b20 tx=0x7f0ae8010000 comp rx=0 tx=0).stop 2023-06-23T18:36:15.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.951+0000 7f0afea5e700 1 -- 172.21.15.79:0/3657194821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0af80b83c0 msgr2=0x7f0af8150360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:15.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.951+0000 7f0afea5e700 1 --2- 172.21.15.79:0/3657194821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0af80b83c0 0x7f0af8150360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000e870 tx=0x7f0ae000ec30 comp rx=0 tx=0).stop 2023-06-23T18:36:15.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.951+0000 7f0afea5e700 1 -- 172.21.15.79:0/3657194821 shutdown_connections 2023-06-23T18:36:15.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.951+0000 7f0afea5e700 1 --2- 172.21.15.79:0/3657194821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0af80d3710 0x7f0af80b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:15.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.951+0000 7f0afea5e700 1 --2- 172.21.15.79:0/3657194821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0af80b83c0 0x7f0af8150360 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:15.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.951+0000 7f0afea5e700 1 --2- 172.21.15.79:0/3657194821 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0ad806ace0 0x7f0ad806d190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:15.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.951+0000 7f0afea5e700 1 --2- 172.21.15.79:0/3657194821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0af80cfa90 0x7f0af80c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:15.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.951+0000 7f0afea5e700 1 -- 172.21.15.79:0/3657194821 >> 172.21.15.79:0/3657194821 conn(0x7f0af806b350 msgr2=0x7f0af80b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:15.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.951+0000 7f0afea5e700 1 -- 172.21.15.79:0/3657194821 shutdown_connections 2023-06-23T18:36:15.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:15.951+0000 7f0afea5e700 1 -- 172.21.15.79:0/3657194821 wait complete. 2023-06-23T18:36:15.968 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-23T18:36:15.968 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-23T18:36:15.968 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-06-23T18:36:15.969 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-06-23T18:36:15.970 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-06-23T18:36:15.970 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-23T18:36:15.970 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-23T18:36:15.970 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-23T18:36:15.970 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-06-23T18:36:15.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:15.971 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-06-23T18:36:15.988 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:36:15.988 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:36:15.989 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:36:15.989 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:15.989 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-06-23T18:36:15.989 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:15.989 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:36:15.989 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:15.989 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:36:15.990 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:15.990 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:36:15.990 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:15.990 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-23T18:36:15.990 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:15.990 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-23T18:36:15.990 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42412sec preferred_lft 42412sec 2023-06-23T18:36:15.991 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::1cf2:57d9:5266:4005/64 scope link noprefixroute 2023-06-23T18:36:15.991 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:15.991 INFO:teuthology.orchestra.run.smithi155.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:36:15.991 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether f6:b7:da:f4:04:16 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:36:15.991 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-23T18:36:15.991 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:15.991 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::74fc:c7ff:fe1c:d1f2/64 scope link 2023-06-23T18:36:15.991 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:15.992 INFO:teuthology.orchestra.run.smithi155.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-23T18:36:15.992 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether f6:b7:da:f4:04:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-06-23T18:36:15.992 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::f4b7:daff:fef4:416/64 scope link tentative 2023-06-23T18:36:15.992 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:15.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:15.993 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-23T18:36:16.054 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-06-23T18:36:16.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:16.055 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-06-23T18:36:16.114 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-06-23T18:36:16.114 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-23T18:36:16.114 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-23T18:36:16.114 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-23T18:36:16.114 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-23T18:36:16.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:16.115 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-06-23T18:36:16.115 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-06-23T18:36:16.116 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-23T18:36:16.116 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-23T18:36:16.117 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-23T18:36:16.117 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-06-23T18:36:16.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:16.219 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-06-23T18:36:16.235 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-06-23T18:36:16.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:16.236 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list-id 2023-06-23T18:36:16.293 INFO:teuthology.orchestra.run.smithi155.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-06-23T18:36:16.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:16.295 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-06-23T18:36:16.295 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-06-23T18:36:16.296 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-06-23T18:36:16.296 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-06-23T18:36:16.297 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-06-23T18:36:16.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:16.424 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-06-23T18:36:16.459 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:36:16.459 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:36:16.460 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:36:16.460 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:16.460 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-06-23T18:36:16.460 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:16.460 INFO:teuthology.orchestra.run.smithi155.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:36:16.460 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 46:a5:b2:68:7b:e5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-23T18:36:16.460 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-23T18:36:16.461 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:16.461 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::44a5:b2ff:fe68:7be5/64 scope link 2023-06-23T18:36:16.461 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:16.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:16.473 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-06-23T18:36:16.548 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:36:16.549 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:36:16.549 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:36:16.549 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:16.549 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-06-23T18:36:16.549 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:16.549 INFO:teuthology.orchestra.run.smithi155.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:36:16.549 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 46:a5:b2:68:7b:e5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-23T18:36:16.550 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-23T18:36:16.550 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:16.550 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::44a5:b2ff:fe68:7be5/64 scope link 2023-06-23T18:36:16.550 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:36:16.569 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-06-23T18:36:16.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:16.569 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-06-23T18:36:16.570 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-06-23T18:36:16.570 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-06-23T18:36:16.571 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-06-23T18:36:16.571 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-06-23T18:36:16.571 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-06-23T18:36:16.571 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-06-23T18:36:16.572 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-06-23T18:36:16.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:16.801 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-06-23T18:36:16.801 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-06-23T18:36:16.802 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.1 up 2023-06-23T18:36:16.802 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set dev brx.1 master ceph-brx 2023-06-23T18:36:16.802 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-06-23T18:36:16.912 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-06-23T18:36:16.913 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:36:16.913 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-06-23T18:36:16.933 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-06-23T18:36:16.933 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:36:16.933 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-06-23T18:36:16.991 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-06-23T18:36:17.059 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:36:17.060 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:36:17.125 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:36:17.126 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:36:17.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:17.127 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-06-23T18:36:17.183 INFO:teuthology.orchestra.run.smithi155.stdout:46 2023-06-23T18:36:17.183 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [46] 2023-06-23T18:36:17.184 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-06-23T18:36:17.226 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-06-23T18:36:17.252 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:36:17.253 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:36:17.311 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi155.stderr:2023-06-23T18:36:17.307+0000 7fb2417855c0 -1 init, newargv = 0x55927b71e0a0 newargc=15 2023-06-23T18:36:17.312 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi155.stderr:ceph-fuse[107975]: starting ceph client 2023-06-23T18:36:17.317 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi155.stderr:ceph-fuse[107975]: starting fuse 2023-06-23T18:36:17.320 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:36:17.321 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:36:17.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:17.322 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-06-23T18:36:17.379 INFO:teuthology.orchestra.run.smithi155.stdout:46 2023-06-23T18:36:17.379 INFO:teuthology.orchestra.run.smithi155.stdout:48 2023-06-23T18:36:17.379 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46, 48] 2023-06-23T18:36:17.380 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-23T18:36:17.380 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-06-23T18:36:17.380 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-06-23T18:36:17.381 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-06-23T18:36:17.381 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-06-23T18:36:17.381 DEBUG:teuthology.orchestra.run.smithi155:> 2023-06-23T18:36:17.381 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-06-23T18:36:17.382 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-06-23T18:36:17.382 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-06-23T18:36:17.382 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-06-23T18:36:17.382 DEBUG:teuthology.orchestra.run.smithi155:> 2023-06-23T18:36:17.383 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-06-23T18:36:17.383 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-06-23T18:36:17.383 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-06-23T18:36:17.383 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-06-23T18:36:17.384 DEBUG:teuthology.orchestra.run.smithi155:> 2023-06-23T18:36:17.384 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-06-23T18:36:17.384 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-23T18:36:17.384 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-23T18:36:17.385 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-23T18:36:17.385 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-06-23T18:36:17.385 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-06-23T18:36:17.385 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-06-23T18:36:17.386 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-23T18:36:17.386 DEBUG:teuthology.orchestra.run.smithi155:> 2023-06-23T18:36:17.386 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.3")) 2023-06-23T18:36:17.386 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-06-23T18:36:17.502 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.3.107975.asok 2023-06-23T18:36:17.685 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.107975.asok 2023-06-23T18:36:17.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:17.686 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.107975.asok status 2023-06-23T18:36:17.841 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-06-23T18:36:17.841 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-06-23T18:36:17.841 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "8627af3e0adcb765a3c249fcc209cba9f4873e1b", 2023-06-23T18:36:17.841 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:36:17.841 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "3", 2023-06-23T18:36:17.842 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-06-23T18:36:17.842 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-06-23T18:36:17.842 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "107975", 2023-06-23T18:36:17.842 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-06-23T18:36:17.842 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-06-23T18:36:17.842 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-06-23T18:36:17.842 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-06-23T18:36:17.843 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 4648, 2023-06-23T18:36:17.843 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-06-23T18:36:17.843 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-06-23T18:36:17.843 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-06-23T18:36:17.844 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 4648 2023-06-23T18:36:17.844 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-06-23T18:36:17.844 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-06-23T18:36:17.844 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-06-23T18:36:17.844 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.2:0", 2023-06-23T18:36:17.844 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 1816553790 2023-06-23T18:36:17.844 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-06-23T18:36:17.845 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-06-23T18:36:17.845 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-06-23T18:36:17.845 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-06-23T18:36:17.845 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.2:0", 2023-06-23T18:36:17.845 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 1816553790 2023-06-23T18:36:17.845 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-06-23T18:36:17.845 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.4648 192.168.0.2:0/1816553790", 2023-06-23T18:36:17.846 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.2:0/1816553790", 2023-06-23T18:36:17.846 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-06-23T18:36:17.846 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 8, 2023-06-23T18:36:17.846 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 20, 2023-06-23T18:36:17.846 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-06-23T18:36:17.846 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-06-23T18:36:17.846 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-06-23T18:36:17.846 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-06-23T18:36:17.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:17.851 DEBUG:teuthology.orchestra.run.smithi079:> stat --file-system '--printf=%T 2023-06-23T18:36:17.851 DEBUG:teuthology.orchestra.run.smithi079:> ' -- /home/ubuntu/cephtest/mnt.0 2023-06-23T18:36:17.873 INFO:teuthology.orchestra.run.smithi079.stdout:fuseblk 2023-06-23T18:36:17.873 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-06-23T18:36:17.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:17.874 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-06-23T18:36:17.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:17.947 DEBUG:teuthology.orchestra.run.smithi079:> stat --file-system '--printf=%T 2023-06-23T18:36:17.947 DEBUG:teuthology.orchestra.run.smithi079:> ' -- /home/ubuntu/cephtest/mnt.2 2023-06-23T18:36:18.008 INFO:teuthology.orchestra.run.smithi079.stdout:fuseblk 2023-06-23T18:36:18.008 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-06-23T18:36:18.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:18.009 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-06-23T18:36:18.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:18.080 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-06-23T18:36:18.080 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2023-06-23T18:36:18.098 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-06-23T18:36:18.098 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-06-23T18:36:18.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:18.099 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-06-23T18:36:18.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:18.167 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-06-23T18:36:18.167 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.3 2023-06-23T18:36:18.222 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-06-23T18:36:18.222 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-06-23T18:36:18.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:18.223 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-06-23T18:36:18.288 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-06-23T18:36:18.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:18.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.431+0000 7f9662a1e700 1 Processor -- start 2023-06-23T18:36:18.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.432+0000 7f9662a1e700 1 -- start start 2023-06-23T18:36:18.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.432+0000 7f9662a1e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f965c159170 0x7f965c0cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:18.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.432+0000 7f9662a1e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f965c0ce7f0 0x7f965c0cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:18.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.432+0000 7f9662a1e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965c15cd50 con 0x7f965c0cd900 2023-06-23T18:36:18.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.432+0000 7f9662a1e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f965c0cdd80 con 0x7f965c159170 2023-06-23T18:36:18.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.432+0000 7f9662a1e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c15cfa0 con 0x7f965c0ce7f0 2023-06-23T18:36:18.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.433+0000 7f965bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f965c159170 0x7f965c0cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:18.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.433+0000 7f965b7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f965c0cd900 0x7f965c0ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43000/0 (socket says 172.21.15.79:43000) 2023-06-23T18:36:18.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.433+0000 7f965bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f965c159170 0x7f965c0cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39958/0 (socket says 172.21.15.79:39958) 2023-06-23T18:36:18.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.433+0000 7f965b7fe700 1 -- 172.21.15.79:0/3005735678 learned_addr learned my addr 172.21.15.79:0/3005735678 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:18.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.433+0000 7f9660fbb700 1 --2- 172.21.15.79:0/3005735678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f965c0ce7f0 0x7f965c0cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:18.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.433+0000 7f965bfff700 1 -- 172.21.15.79:0/3005735678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f965c0ce7f0 msgr2=0x7f965c0cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:18.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.433+0000 7f965bfff700 1 --2- 172.21.15.79:0/3005735678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f965c0ce7f0 0x7f965c0cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.433+0000 7f965bfff700 1 -- 172.21.15.79:0/3005735678 >> v1:172.21.15.79:6789/0 conn(0x7f965c0cd900 legacy=0x7f965c0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:18.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.433+0000 7f965bfff700 1 -- 172.21.15.79:0/3005735678 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c15d280 con 0x7f965c159170 2023-06-23T18:36:18.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.433+0000 7f965affd700 1 -- 172.21.15.79:0/3005735678 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371290368 0 0) 0x7f965c15cd50 con 0x7f965c0cd900 2023-06-23T18:36:18.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.433+0000 7f965bfff700 1 --2- 172.21.15.79:0/3005735678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f965c159170 0x7f965c0cd3c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f964c00bae0 tx=0x7f964c00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310383a2f76338f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:18.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.433+0000 7f965affd700 1 -- 172.21.15.79:0/3005735678 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c00dd30 con 0x7f965c159170 2023-06-23T18:36:18.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.433+0000 7f965affd700 1 -- 172.21.15.79:0/3005735678 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c00de90 con 0x7f965c159170 2023-06-23T18:36:18.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.433+0000 7f965affd700 1 -- 172.21.15.79:0/3005735678 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c010690 con 0x7f965c159170 2023-06-23T18:36:18.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.434+0000 7f9662a1e700 1 -- 172.21.15.79:0/3005735678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f965c159170 msgr2=0x7f965c0cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:18.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.434+0000 7f9662a1e700 1 --2- 172.21.15.79:0/3005735678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f965c159170 0x7f965c0cd3c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f964c00bae0 tx=0x7f964c00bdf0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.434+0000 7f9662a1e700 1 -- 172.21.15.79:0/3005735678 shutdown_connections 2023-06-23T18:36:18.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.434+0000 7f9662a1e700 1 --2- 172.21.15.79:0/3005735678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f965c0ce7f0 0x7f965c0cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.434+0000 7f9662a1e700 1 --2- 172.21.15.79:0/3005735678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f965c159170 0x7f965c0cd3c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.434+0000 7f9662a1e700 1 -- 172.21.15.79:0/3005735678 >> 172.21.15.79:0/3005735678 conn(0x7f965c06d660 msgr2=0x7f965c06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:18.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.434+0000 7f9662a1e700 1 -- 172.21.15.79:0/3005735678 shutdown_connections 2023-06-23T18:36:18.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.434+0000 7f9662a1e700 1 -- 172.21.15.79:0/3005735678 wait complete. 2023-06-23T18:36:18.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.435+0000 7f9662a1e700 1 Processor -- start 2023-06-23T18:36:18.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.435+0000 7f9662a1e700 1 -- start start 2023-06-23T18:36:18.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9662a1e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f965c0cd900 0x7f965c0b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:18.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9662a1e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f965c0ce7f0 0x7f965c0c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:18.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9662a1e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f965c159170 0x7f965c1572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9662a1e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c0b4890 con 0x7f965c159170 2023-06-23T18:36:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9662a1e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c0c5190 con 0x7f965c0cd900 2023-06-23T18:36:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9662a1e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c0c5470 con 0x7f965c0ce7f0 2023-06-23T18:36:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9660fbb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f965c159170 0x7f965c1572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9660fbb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f965c159170 0x7f965c1572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39966/0 (socket says 172.21.15.79:39966) 2023-06-23T18:36:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9660fbb700 1 -- 172.21.15.79:0/303827451 learned_addr learned my addr 172.21.15.79:0/303827451 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f965bfff700 1 --2- 172.21.15.79:0/303827451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f965c0cd900 0x7f965c0b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:18.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f965b7fe700 1 --2- 172.21.15.79:0/303827451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f965c0ce7f0 0x7f965c0c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:18.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9660fbb700 1 -- 172.21.15.79:0/303827451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f965c0ce7f0 msgr2=0x7f965c0c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:18.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9660fbb700 1 --2- 172.21.15.79:0/303827451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f965c0ce7f0 0x7f965c0c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9660fbb700 1 -- 172.21.15.79:0/303827451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f965c0cd900 msgr2=0x7f965c0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:18.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9660fbb700 1 --2- 172.21.15.79:0/303827451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f965c0cd900 0x7f965c0b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9660fbb700 1 -- 172.21.15.79:0/303827451 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c00b740 con 0x7f965c159170 2023-06-23T18:36:18.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.436+0000 7f9660fbb700 1 --2- 172.21.15.79:0/303827451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f965c159170 0x7f965c1572f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9654007e40 tx=0x7f965400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:18.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.437+0000 7f9658ff9700 1 -- 172.21.15.79:0/303827451 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965400ed30 con 0x7f965c159170 2023-06-23T18:36:18.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.437+0000 7f9658ff9700 1 -- 172.21.15.79:0/303827451 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9654014420 con 0x7f965c159170 2023-06-23T18:36:18.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.437+0000 7f9662a1e700 1 -- 172.21.15.79:0/303827451 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965c0c5750 con 0x7f965c159170 2023-06-23T18:36:18.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.437+0000 7f9658ff9700 1 -- 172.21.15.79:0/303827451 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540147d0 con 0x7f965c159170 2023-06-23T18:36:18.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.437+0000 7f9662a1e700 1 -- 172.21.15.79:0/303827451 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965c0c5d10 con 0x7f965c159170 2023-06-23T18:36:18.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.438+0000 7f9662a1e700 1 -- 172.21.15.79:0/303827451 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965c06e280 con 0x7f965c159170 2023-06-23T18:36:18.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.438+0000 7f9658ff9700 1 -- 172.21.15.79:0/303827451 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9654014930 con 0x7f965c159170 2023-06-23T18:36:18.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.439+0000 7f9658ff9700 1 --2- 172.21.15.79:0/303827451 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f963c06acb0 0x7f963c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:18.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.439+0000 7f9658ff9700 1 -- 172.21.15.79:0/303827451 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9654089b20 con 0x7f965c159170 2023-06-23T18:36:18.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.439+0000 7f965bfff700 1 --2- 172.21.15.79:0/303827451 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f963c06acb0 0x7f963c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:18.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.439+0000 7f965bfff700 1 --2- 172.21.15.79:0/303827451 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f963c06acb0 0x7f963c06d160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f964c00be30 tx=0x7f964c013000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:18.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.443+0000 7f9658ff9700 1 -- 172.21.15.79:0/303827451 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9654054bd0 con 0x7f965c159170 2023-06-23T18:36:18.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.629+0000 7f9662a1e700 1 -- 172.21.15.79:0/303827451 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f965c0b5770 con 0x7f965c159170 2023-06-23T18:36:18.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.630+0000 7f9658ff9700 1 -- 172.21.15.79:0/303827451 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7f96540581f0 con 0x7f965c159170 2023-06-23T18:36:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:18.636 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:18.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.632+0000 7f9662a1e700 1 -- 172.21.15.79:0/303827451 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f963c06acb0 msgr2=0x7f963c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:18.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.632+0000 7f9662a1e700 1 --2- 172.21.15.79:0/303827451 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f963c06acb0 0x7f963c06d160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f964c00be30 tx=0x7f964c013000 comp rx=0 tx=0).stop 2023-06-23T18:36:18.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.632+0000 7f9662a1e700 1 -- 172.21.15.79:0/303827451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f965c159170 msgr2=0x7f965c1572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:18.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.632+0000 7f9662a1e700 1 --2- 172.21.15.79:0/303827451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f965c159170 0x7f965c1572f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9654007e40 tx=0x7f965400e3c0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.632+0000 7f9662a1e700 1 -- 172.21.15.79:0/303827451 shutdown_connections 2023-06-23T18:36:18.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.633+0000 7f9662a1e700 1 --2- 172.21.15.79:0/303827451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f965c159170 0x7f965c1572f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.633+0000 7f9662a1e700 1 --2- 172.21.15.79:0/303827451 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f963c06acb0 0x7f963c06d160 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.633+0000 7f9662a1e700 1 --2- 172.21.15.79:0/303827451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f965c0cd900 0x7f965c0b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.633+0000 7f9662a1e700 1 --2- 172.21.15.79:0/303827451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f965c0ce7f0 0x7f965c0c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.633+0000 7f9662a1e700 1 -- 172.21.15.79:0/303827451 >> 172.21.15.79:0/303827451 conn(0x7f965c06d660 msgr2=0x7f965c0af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:18.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.633+0000 7f9662a1e700 1 -- 172.21.15.79:0/303827451 shutdown_connections 2023-06-23T18:36:18.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.633+0000 7f9662a1e700 1 -- 172.21.15.79:0/303827451 wait complete. 2023-06-23T18:36:18.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:18.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:18.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.768+0000 7fbd179e1700 1 Processor -- start 2023-06-23T18:36:18.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.768+0000 7fbd179e1700 1 -- start start 2023-06-23T18:36:18.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd179e1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbd101438f0 0x7fbd100b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:18.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd179e1700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbd100d08c0 0x7fbd100d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:18.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd179e1700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd100d1200 con 0x7fbd100cedf0 2023-06-23T18:36:18.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd179e1700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd100b0b10 con 0x7fbd101438f0 2023-06-23T18:36:18.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd179e1700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd100b0df0 con 0x7fbd100d08c0 2023-06-23T18:36:18.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd14f7c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbd101438f0 0x7fbd100b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:18.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd14f7c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbd101438f0 0x7fbd100b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39972/0 (socket says 172.21.15.79:39972) 2023-06-23T18:36:18.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd1577d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fbd100cedf0 0x7fbd100b02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43012/0 (socket says 172.21.15.79:43012) 2023-06-23T18:36:18.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd14f7c700 1 -- 172.21.15.79:0/2529784358 learned_addr learned my addr 172.21.15.79:0/2529784358 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:18.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd14f7c700 1 -- 172.21.15.79:0/2529784358 >> v2:172.21.15.155:3300/0 conn(0x7fbd100d08c0 msgr2=0x7fbd100d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:18.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd15f7e700 1 --2- 172.21.15.79:0/2529784358 >> v2:172.21.15.155:3300/0 conn(0x7fbd100d08c0 0x7fbd100d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:18.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd14f7c700 1 --2- 172.21.15.79:0/2529784358 >> v2:172.21.15.155:3300/0 conn(0x7fbd100d08c0 0x7fbd100d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd14f7c700 1 -- 172.21.15.79:0/2529784358 >> v1:172.21.15.79:6789/0 conn(0x7fbd100cedf0 legacy=0x7fbd100b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:18.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd07fff700 1 -- 172.21.15.79:0/2529784358 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393143796 0 0) 0x7fbd100d1200 con 0x7fbd100cedf0 2023-06-23T18:36:18.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd14f7c700 1 -- 172.21.15.79:0/2529784358 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd100b10d0 con 0x7fbd101438f0 2023-06-23T18:36:18.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd15f7e700 1 --2- 172.21.15.79:0/2529784358 >> v2:172.21.15.155:3300/0 conn(0x7fbd100d08c0 0x7fbd100d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:18.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd14f7c700 1 --2- 172.21.15.79:0/2529784358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd101438f0 0x7fbd100b1bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd100668b0 tx=0x7fbd00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161b1ad4c29118e6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:18.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd07fff700 1 -- 172.21.15.79:0/2529784358 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0000b870 con 0x7fbd101438f0 2023-06-23T18:36:18.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd07fff700 1 -- 172.21.15.79:0/2529784358 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0000b9d0 con 0x7fbd101438f0 2023-06-23T18:36:18.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.769+0000 7fbd07fff700 1 -- 172.21.15.79:0/2529784358 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0000bd60 con 0x7fbd101438f0 2023-06-23T18:36:18.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.770+0000 7fbd179e1700 1 -- 172.21.15.79:0/2529784358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd101438f0 msgr2=0x7fbd100b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:18.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.770+0000 7fbd179e1700 1 --2- 172.21.15.79:0/2529784358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd101438f0 0x7fbd100b1bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd100668b0 tx=0x7fbd00009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.770+0000 7fbd179e1700 1 -- 172.21.15.79:0/2529784358 shutdown_connections 2023-06-23T18:36:18.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.770+0000 7fbd179e1700 1 --2- 172.21.15.79:0/2529784358 >> v2:172.21.15.155:3300/0 conn(0x7fbd100d08c0 0x7fbd100d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.770+0000 7fbd179e1700 1 --2- 172.21.15.79:0/2529784358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd101438f0 0x7fbd100b1bb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.770+0000 7fbd179e1700 1 -- 172.21.15.79:0/2529784358 >> 172.21.15.79:0/2529784358 conn(0x7fbd1006a7b0 msgr2=0x7fbd10099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:18.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.770+0000 7fbd179e1700 1 -- 172.21.15.79:0/2529784358 shutdown_connections 2023-06-23T18:36:18.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.770+0000 7fbd179e1700 1 -- 172.21.15.79:0/2529784358 wait complete. 2023-06-23T18:36:18.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.771+0000 7fbd179e1700 1 Processor -- start 2023-06-23T18:36:18.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.771+0000 7fbd179e1700 1 -- start start 2023-06-23T18:36:18.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd179e1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd100cedf0 0x7fbd100c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:18.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd179e1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100d08c0 0x7fbd101583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:18.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd179e1700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd101438f0 0x7fbd1009a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:18.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd179e1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1009d2c0 con 0x7fbd100d08c0 2023-06-23T18:36:18.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd179e1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd10158900 con 0x7fbd100cedf0 2023-06-23T18:36:18.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd179e1700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd100c4620 con 0x7fbd101438f0 2023-06-23T18:36:18.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd14f7c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100d08c0 0x7fbd101583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:18.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd14f7c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100d08c0 0x7fbd101583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39978/0 (socket says 172.21.15.79:39978) 2023-06-23T18:36:18.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd14f7c700 1 -- 172.21.15.79:0/1334248831 learned_addr learned my addr 172.21.15.79:0/1334248831 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:18.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd15f7e700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd101438f0 0x7fbd1009a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:18.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd1577d700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd100cedf0 0x7fbd100c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:18.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd14f7c700 1 -- 172.21.15.79:0/1334248831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd101438f0 msgr2=0x7fbd1009a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:18.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd14f7c700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd101438f0 0x7fbd1009a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd14f7c700 1 -- 172.21.15.79:0/1334248831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd100cedf0 msgr2=0x7fbd100c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:18.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd14f7c700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd100cedf0 0x7fbd100c4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd14f7c700 1 -- 172.21.15.79:0/1334248831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd00009550 con 0x7fbd100d08c0 2023-06-23T18:36:18.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.772+0000 7fbd15f7e700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd101438f0 0x7fbd1009a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:18.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.773+0000 7fbd14f7c700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100d08c0 0x7fbd101583c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbd00014040 tx=0x7fbd00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:18.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.773+0000 7fbd05ffb700 1 -- 172.21.15.79:0/1334248831 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd00004310 con 0x7fbd100d08c0 2023-06-23T18:36:18.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.773+0000 7fbd05ffb700 1 -- 172.21.15.79:0/1334248831 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0001f070 con 0x7fbd100d08c0 2023-06-23T18:36:18.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.773+0000 7fbd179e1700 1 -- 172.21.15.79:0/1334248831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd100c4900 con 0x7fbd100d08c0 2023-06-23T18:36:18.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.773+0000 7fbd05ffb700 1 -- 172.21.15.79:0/1334248831 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd000036a0 con 0x7fbd100d08c0 2023-06-23T18:36:18.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.773+0000 7fbd179e1700 1 -- 172.21.15.79:0/1334248831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd1009cc70 con 0x7fbd100d08c0 2023-06-23T18:36:18.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.774+0000 7fbd179e1700 1 -- 172.21.15.79:0/1334248831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd100672c0 con 0x7fbd100d08c0 2023-06-23T18:36:18.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.774+0000 7fbd05ffb700 1 -- 172.21.15.79:0/1334248831 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbd00021030 con 0x7fbd100d08c0 2023-06-23T18:36:18.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.775+0000 7fbd05ffb700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbcf406ace0 0x7fbcf406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:18.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.775+0000 7fbd05ffb700 1 -- 172.21.15.79:0/1334248831 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbd0008a8f0 con 0x7fbd100d08c0 2023-06-23T18:36:18.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.775+0000 7fbd1577d700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbcf406ace0 0x7fbcf406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:18.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.775+0000 7fbd1577d700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbcf406ace0 0x7fbcf406d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0016f0 tx=0x7fbd0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:18.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.779+0000 7fbd05ffb700 1 -- 172.21.15.79:0/1334248831 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbd00055ab0 con 0x7fbd100d08c0 2023-06-23T18:36:18.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.963+0000 7fbd179e1700 1 -- 172.21.15.79:0/1334248831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbd1015d7b0 con 0x7fbd100d08c0 2023-06-23T18:36:18.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.963+0000 7fbd05ffb700 1 -- 172.21.15.79:0/1334248831 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbd000590d0 con 0x7fbd100d08c0 2023-06-23T18:36:18.968 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:18.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.965+0000 7fbd179e1700 1 -- 172.21.15.79:0/1334248831 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbcf406ace0 msgr2=0x7fbcf406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:18.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.965+0000 7fbd179e1700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbcf406ace0 0x7fbcf406d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0016f0 tx=0x7fbd0c009040 comp rx=0 tx=0).stop 2023-06-23T18:36:18.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.965+0000 7fbd179e1700 1 -- 172.21.15.79:0/1334248831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100d08c0 msgr2=0x7fbd101583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:18.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.965+0000 7fbd179e1700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100d08c0 0x7fbd101583c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbd00014040 tx=0x7fbd00004750 comp rx=0 tx=0).stop 2023-06-23T18:36:18.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.965+0000 7fbd179e1700 1 -- 172.21.15.79:0/1334248831 shutdown_connections 2023-06-23T18:36:18.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.965+0000 7fbd179e1700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd101438f0 0x7fbd1009a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.965+0000 7fbd179e1700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbcf406ace0 0x7fbcf406d190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.965+0000 7fbd179e1700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd100cedf0 0x7fbd100c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.965+0000 7fbd179e1700 1 --2- 172.21.15.79:0/1334248831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100d08c0 0x7fbd101583c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:18.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.965+0000 7fbd179e1700 1 -- 172.21.15.79:0/1334248831 >> 172.21.15.79:0/1334248831 conn(0x7fbd1006a7b0 msgr2=0x7fbd1006f510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:18.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.965+0000 7fbd179e1700 1 -- 172.21.15.79:0/1334248831 shutdown_connections 2023-06-23T18:36:18.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:18.965+0000 7fbd179e1700 1 -- 172.21.15.79:0/1334248831 wait complete. 2023-06-23T18:36:18.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:19.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.099+0000 7fbe3495a700 1 Processor -- start 2023-06-23T18:36:19.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.099+0000 7fbe3495a700 1 -- start start 2023-06-23T18:36:19.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.100+0000 7fbe3495a700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbe300cd440 0x7fbe301608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.100+0000 7fbe3495a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe300cdc90 con 0x7fbe301569c0 2023-06-23T18:36:19.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.100+0000 7fbe3495a700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3015d0f0 con 0x7fbe3006a5a0 2023-06-23T18:36:19.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.100+0000 7fbe3495a700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe300cf570 con 0x7fbe300cd440 2023-06-23T18:36:19.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2e59c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fbe301569c0 0x7fbe30069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43026/0 (socket says 172.21.15.79:43026) 2023-06-23T18:36:19.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2e59c700 1 -- 172.21.15.79:0/1278383322 learned_addr learned my addr 172.21.15.79:0/1278383322 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:19.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 -- 172.21.15.79:0/1278383322 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945525822 0 0) 0x7fbe20001110 con 0x7fbe301569c0 2023-06-23T18:36:19.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 -- 172.21.15.79:0/1278383322 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe180035b0 con 0x7fbe301569c0 2023-06-23T18:36:19.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2ed9d700 1 --2- 172.21.15.79:0/1278383322 >> v2:172.21.15.155:3300/0 conn(0x7fbe300cd440 0x7fbe301608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 -- 172.21.15.79:0/1278383322 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 481311132 0 0) 0x7fbe180035b0 con 0x7fbe301569c0 2023-06-23T18:36:19.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 -- 172.21.15.79:0/1278383322 >> v2:172.21.15.155:3300/0 conn(0x7fbe300cd440 msgr2=0x7fbe301608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:19.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 --2- 172.21.15.79:0/1278383322 >> v2:172.21.15.155:3300/0 conn(0x7fbe300cd440 0x7fbe301608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 -- 172.21.15.79:0/1278383322 >> v1:172.21.15.155:6789/0 conn(0x7fbe3006a5a0 legacy=0x7fbe300b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 -- 172.21.15.79:0/1278383322 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe30156d90 con 0x7fbe301569c0 2023-06-23T18:36:19.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 -- 172.21.15.79:0/1278383322 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fbe20001800 con 0x7fbe301569c0 2023-06-23T18:36:19.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 -- 172.21.15.79:0/1278383322 >> v1:172.21.15.79:6789/0 conn(0x7fbe301569c0 legacy=0x7fbe30069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 --2- 172.21.15.79:0/1278383322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe18003a00 0x7fbe18003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 --2- 172.21.15.79:0/1278383322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe18004850 0x7fbe18004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 --2- 172.21.15.79:0/1278383322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe18005200 0x7fbe1800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 -- 172.21.15.79:0/1278383322 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1800c050 con 0x7fbe18005200 2023-06-23T18:36:19.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 -- 172.21.15.79:0/1278383322 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1800c1c0 con 0x7fbe18004850 2023-06-23T18:36:19.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2d59a700 1 -- 172.21.15.79:0/1278383322 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1800c380 con 0x7fbe18003a00 2023-06-23T18:36:19.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2ed9d700 1 --2- 172.21.15.79:0/1278383322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe18005200 0x7fbe1800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2e59c700 1 --2- 172.21.15.79:0/1278383322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe18003a00 0x7fbe18003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2dd9b700 1 --2- 172.21.15.79:0/1278383322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe18004850 0x7fbe18004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2ed9d700 1 -- 172.21.15.79:0/1278383322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe18003a00 msgr2=0x7fbe18003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:19.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2ed9d700 1 --2- 172.21.15.79:0/1278383322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe18003a00 0x7fbe18003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2ed9d700 1 -- 172.21.15.79:0/1278383322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe18004850 msgr2=0x7fbe18004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:19.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2ed9d700 1 --2- 172.21.15.79:0/1278383322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe18004850 0x7fbe18004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2ed9d700 1 -- 172.21.15.79:0/1278383322 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe180035b0 con 0x7fbe18005200 2023-06-23T18:36:19.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.101+0000 7fbe2dd9b700 1 --2- 172.21.15.79:0/1278383322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe18004850 0x7fbe18004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:19.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.102+0000 7fbe2ed9d700 1 -- 172.21.15.79:0/1278383322 reap_dead start 2023-06-23T18:36:19.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.102+0000 7fbe2ed9d700 1 --2- 172.21.15.79:0/1278383322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe18005200 0x7fbe1800ba10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009fc0 tx=0x7fbe18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4ddc2aa42e76da server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:19.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.102+0000 7fbe2d59a700 1 -- 172.21.15.79:0/1278383322 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28003a20 con 0x7fbe18005200 2023-06-23T18:36:19.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.102+0000 7fbe2d59a700 1 -- 172.21.15.79:0/1278383322 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2800f040 con 0x7fbe18005200 2023-06-23T18:36:19.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.102+0000 7fbe2d59a700 1 -- 172.21.15.79:0/1278383322 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28003de0 con 0x7fbe18005200 2023-06-23T18:36:19.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.102+0000 7fbe3495a700 1 -- 172.21.15.79:0/1278383322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe18005200 msgr2=0x7fbe1800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.102+0000 7fbe3495a700 1 --2- 172.21.15.79:0/1278383322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe18005200 0x7fbe1800ba10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009fc0 tx=0x7fbe18005880 comp rx=0 tx=0).stop 2023-06-23T18:36:19.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.102+0000 7fbe3495a700 1 -- 172.21.15.79:0/1278383322 shutdown_connections 2023-06-23T18:36:19.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.102+0000 7fbe3495a700 1 --2- 172.21.15.79:0/1278383322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe18005200 0x7fbe1800ba10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.102+0000 7fbe3495a700 1 -- 172.21.15.79:0/1278383322 >> 172.21.15.79:0/1278383322 conn(0x7fbe3006afa0 msgr2=0x7fbe3006b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:19.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.102+0000 7fbe3495a700 1 -- 172.21.15.79:0/1278383322 shutdown_connections 2023-06-23T18:36:19.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.102+0000 7fbe3495a700 1 -- 172.21.15.79:0/1278383322 wait complete. 2023-06-23T18:36:19.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.104+0000 7fbe3495a700 1 Processor -- start 2023-06-23T18:36:19.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.104+0000 7fbe3495a700 1 -- start start 2023-06-23T18:36:19.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.104+0000 7fbe3495a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe18005200 0x7fbe300c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.104+0000 7fbe3495a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe300c7cf0 0x7fbe3015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.104+0000 7fbe3495a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe300c8160 0x7fbe3015e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.104+0000 7fbe3495a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300c85d0 con 0x7fbe300c7cf0 2023-06-23T18:36:19.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.104+0000 7fbe3495a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300c88b0 con 0x7fbe300c8160 2023-06-23T18:36:19.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.104+0000 7fbe3495a700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300c8b90 con 0x7fbe18005200 2023-06-23T18:36:19.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe2dd9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe300c7cf0 0x7fbe3015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe2dd9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe300c7cf0 0x7fbe3015db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39998/0 (socket says 172.21.15.79:39998) 2023-06-23T18:36:19.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe2dd9b700 1 -- 172.21.15.79:0/184034463 learned_addr learned my addr 172.21.15.79:0/184034463 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:19.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe2e59c700 1 --2- 172.21.15.79:0/184034463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe18005200 0x7fbe300c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe2ed9d700 1 --2- 172.21.15.79:0/184034463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe300c8160 0x7fbe3015e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe2dd9b700 1 -- 172.21.15.79:0/184034463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe18005200 msgr2=0x7fbe300c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe2dd9b700 1 --2- 172.21.15.79:0/184034463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe18005200 0x7fbe300c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe2dd9b700 1 -- 172.21.15.79:0/184034463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe300c8160 msgr2=0x7fbe3015e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe2dd9b700 1 --2- 172.21.15.79:0/184034463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe300c8160 0x7fbe3015e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe2dd9b700 1 -- 172.21.15.79:0/184034463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe300c8e70 con 0x7fbe300c7cf0 2023-06-23T18:36:19.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe2dd9b700 1 --2- 172.21.15.79:0/184034463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe300c7cf0 0x7fbe3015db20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400e460 tx=0x7fbe2400e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:19.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe1effd700 1 -- 172.21.15.79:0/184034463 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe240052e0 con 0x7fbe300c7cf0 2023-06-23T18:36:19.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe1effd700 1 -- 172.21.15.79:0/184034463 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe24005440 con 0x7fbe300c7cf0 2023-06-23T18:36:19.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe3495a700 1 -- 172.21.15.79:0/184034463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe300c9150 con 0x7fbe300c7cf0 2023-06-23T18:36:19.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe1effd700 1 -- 172.21.15.79:0/184034463 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24003880 con 0x7fbe300c7cf0 2023-06-23T18:36:19.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.105+0000 7fbe3495a700 1 -- 172.21.15.79:0/184034463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe30147d20 con 0x7fbe300c7cf0 2023-06-23T18:36:19.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.107+0000 7fbe1effd700 1 -- 172.21.15.79:0/184034463 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbe240039e0 con 0x7fbe300c7cf0 2023-06-23T18:36:19.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.107+0000 7fbe3495a700 1 -- 172.21.15.79:0/184034463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe300cf640 con 0x7fbe300c7cf0 2023-06-23T18:36:19.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.107+0000 7fbe1effd700 1 --2- 172.21.15.79:0/184034463 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbe1006ace0 0x7fbe1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.107+0000 7fbe2e59c700 1 --2- 172.21.15.79:0/184034463 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbe1006ace0 0x7fbe1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.107+0000 7fbe1effd700 1 -- 172.21.15.79:0/184034463 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe24088ae0 con 0x7fbe300c7cf0 2023-06-23T18:36:19.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.107+0000 7fbe2e59c700 1 --2- 172.21.15.79:0/184034463 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbe1006ace0 0x7fbe1006d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbe200012b0 tx=0x7fbe20009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:19.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.111+0000 7fbe1effd700 1 -- 172.21.15.79:0/184034463 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbe24053c40 con 0x7fbe300c7cf0 2023-06-23T18:36:19.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.297+0000 7fbe3495a700 1 -- 172.21.15.79:0/184034463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe300672c0 con 0x7fbe300c7cf0 2023-06-23T18:36:19.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.297+0000 7fbe1effd700 1 -- 172.21.15.79:0/184034463 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe240190a0 con 0x7fbe300c7cf0 2023-06-23T18:36:19.303 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:19.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.300+0000 7fbe3495a700 1 -- 172.21.15.79:0/184034463 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbe1006ace0 msgr2=0x7fbe1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.300+0000 7fbe3495a700 1 --2- 172.21.15.79:0/184034463 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbe1006ace0 0x7fbe1006d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbe200012b0 tx=0x7fbe20009040 comp rx=0 tx=0).stop 2023-06-23T18:36:19.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.300+0000 7fbe3495a700 1 -- 172.21.15.79:0/184034463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe300c7cf0 msgr2=0x7fbe3015db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.300+0000 7fbe3495a700 1 --2- 172.21.15.79:0/184034463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe300c7cf0 0x7fbe3015db20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400e460 tx=0x7fbe2400e820 comp rx=0 tx=0).stop 2023-06-23T18:36:19.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.300+0000 7fbe3495a700 1 -- 172.21.15.79:0/184034463 shutdown_connections 2023-06-23T18:36:19.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.300+0000 7fbe3495a700 1 --2- 172.21.15.79:0/184034463 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbe1006ace0 0x7fbe1006d190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.300+0000 7fbe3495a700 1 --2- 172.21.15.79:0/184034463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe300c8160 0x7fbe3015e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.300+0000 7fbe3495a700 1 --2- 172.21.15.79:0/184034463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe18005200 0x7fbe300c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.300+0000 7fbe3495a700 1 --2- 172.21.15.79:0/184034463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe300c7cf0 0x7fbe3015db20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.300+0000 7fbe3495a700 1 -- 172.21.15.79:0/184034463 >> 172.21.15.79:0/184034463 conn(0x7fbe3006afa0 msgr2=0x7fbe3006dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:19.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.300+0000 7fbe3495a700 1 -- 172.21.15.79:0/184034463 shutdown_connections 2023-06-23T18:36:19.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.300+0000 7fbe3495a700 1 -- 172.21.15.79:0/184034463 wait complete. 2023-06-23T18:36:19.317 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:19.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.434+0000 7f9f2e265700 1 Processor -- start 2023-06-23T18:36:19.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.434+0000 7f9f2e265700 1 -- start start 2023-06-23T18:36:19.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.435+0000 7f9f2e265700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9f28159220 0x7f9f280ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.435+0000 7f9f2e265700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9f280cc3d0 0x7f9f280cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.435+0000 7f9f2e265700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f280cf600 con 0x7f9f280703a0 2023-06-23T18:36:19.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.435+0000 7f9f2e265700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f280ccf40 con 0x7f9f280cc3d0 2023-06-23T18:36:19.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.435+0000 7f9f2e265700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f280c9b00 con 0x7f9f28159220 2023-06-23T18:36:19.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.435+0000 7f9f277fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9f280cc3d0 0x7f9f280cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.435+0000 7f9f277fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9f280cc3d0 0x7f9f280cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40004/0 (socket says 172.21.15.79:40004) 2023-06-23T18:36:19.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.435+0000 7f9f277fe700 1 -- 172.21.15.79:0/3104776890 learned_addr learned my addr 172.21.15.79:0/3104776890 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:19.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.436+0000 7f9f27fff700 1 --2- 172.21.15.79:0/3104776890 >> v2:172.21.15.155:3300/0 conn(0x7f9f28159220 0x7f9f280ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.436+0000 7f9f277fe700 1 -- 172.21.15.79:0/3104776890 >> v2:172.21.15.155:3300/0 conn(0x7f9f28159220 msgr2=0x7f9f280ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:19.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.436+0000 7f9f277fe700 1 --2- 172.21.15.79:0/3104776890 >> v2:172.21.15.155:3300/0 conn(0x7f9f28159220 0x7f9f280ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.436+0000 7f9f277fe700 1 -- 172.21.15.79:0/3104776890 >> v1:172.21.15.79:6789/0 conn(0x7f9f280703a0 legacy=0x7f9f280707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.436+0000 7f9f277fe700 1 -- 172.21.15.79:0/3104776890 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f280c9de0 con 0x7f9f280cc3d0 2023-06-23T18:36:19.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.436+0000 7f9f277fe700 1 --2- 172.21.15.79:0/3104776890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f280cc3d0 0x7f9f280cc870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9f280710d0 tx=0x7f9f18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adebb2b5e2978a33 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:19.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.436+0000 7f9f26ffd700 1 -- 172.21.15.79:0/3104776890 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800b870 con 0x7f9f280cc3d0 2023-06-23T18:36:19.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.436+0000 7f9f26ffd700 1 -- 172.21.15.79:0/3104776890 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1800b9d0 con 0x7f9f280cc3d0 2023-06-23T18:36:19.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.436+0000 7f9f26ffd700 1 -- 172.21.15.79:0/3104776890 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800bd60 con 0x7f9f280cc3d0 2023-06-23T18:36:19.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.436+0000 7f9f2e265700 1 -- 172.21.15.79:0/3104776890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f280cc3d0 msgr2=0x7f9f280cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.436+0000 7f9f2e265700 1 --2- 172.21.15.79:0/3104776890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f280cc3d0 0x7f9f280cc870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9f280710d0 tx=0x7f9f18009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.437+0000 7f9f2e265700 1 -- 172.21.15.79:0/3104776890 shutdown_connections 2023-06-23T18:36:19.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.437+0000 7f9f2e265700 1 --2- 172.21.15.79:0/3104776890 >> v2:172.21.15.155:3300/0 conn(0x7f9f28159220 0x7f9f280ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.437+0000 7f9f2e265700 1 --2- 172.21.15.79:0/3104776890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f280cc3d0 0x7f9f280cc870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.437+0000 7f9f2e265700 1 -- 172.21.15.79:0/3104776890 >> 172.21.15.79:0/3104776890 conn(0x7f9f280cbf10 msgr2=0x7f9f280ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:19.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.437+0000 7f9f2e265700 1 -- 172.21.15.79:0/3104776890 shutdown_connections 2023-06-23T18:36:19.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.437+0000 7f9f2e265700 1 -- 172.21.15.79:0/3104776890 wait complete. 2023-06-23T18:36:19.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.438+0000 7f9f2e265700 1 Processor -- start 2023-06-23T18:36:19.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.438+0000 7f9f2e265700 1 -- start start 2023-06-23T18:36:19.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2e265700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f280703a0 0x7f9f2814e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2e265700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f280cc3d0 0x7f9f281e4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2e265700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f28159220 0x7f9f280afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2e265700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280aec60 con 0x7f9f28159220 2023-06-23T18:36:19.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2e265700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280aef40 con 0x7f9f280cc3d0 2023-06-23T18:36:19.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2e265700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280af220 con 0x7f9f280703a0 2023-06-23T18:36:19.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2c802700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f28159220 0x7f9f280afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2c802700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f28159220 0x7f9f280afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40014/0 (socket says 172.21.15.79:40014) 2023-06-23T18:36:19.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2c802700 1 -- 172.21.15.79:0/525874457 learned_addr learned my addr 172.21.15.79:0/525874457 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:19.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f277fe700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f280cc3d0 0x7f9f281e4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f27fff700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f280703a0 0x7f9f2814e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2c802700 1 -- 172.21.15.79:0/525874457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f280703a0 msgr2=0x7f9f2814e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2c802700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f280703a0 0x7f9f2814e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2c802700 1 -- 172.21.15.79:0/525874457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f280cc3d0 msgr2=0x7f9f281e4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2c802700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f280cc3d0 0x7f9f281e4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2c802700 1 -- 172.21.15.79:0/525874457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f18009550 con 0x7f9f28159220 2023-06-23T18:36:19.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f27fff700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f280703a0 0x7f9f2814e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:19.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2c802700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f28159220 0x7f9f280afd00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9f200021c0 tx=0x7f9f200021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:19.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f24ff9700 1 -- 172.21.15.79:0/525874457 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20003c00 con 0x7f9f28159220 2023-06-23T18:36:19.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f24ff9700 1 -- 172.21.15.79:0/525874457 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f20003d60 con 0x7f9f28159220 2023-06-23T18:36:19.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f2e265700 1 -- 172.21.15.79:0/525874457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f280af500 con 0x7f9f28159220 2023-06-23T18:36:19.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.439+0000 7f9f24ff9700 1 -- 172.21.15.79:0/525874457 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20005310 con 0x7f9f28159220 2023-06-23T18:36:19.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.440+0000 7f9f2e265700 1 -- 172.21.15.79:0/525874457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2814ef50 con 0x7f9f28159220 2023-06-23T18:36:19.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.440+0000 7f9f2e265700 1 -- 172.21.15.79:0/525874457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2806c380 con 0x7f9f28159220 2023-06-23T18:36:19.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.441+0000 7f9f24ff9700 1 -- 172.21.15.79:0/525874457 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9f20005470 con 0x7f9f28159220 2023-06-23T18:36:19.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.441+0000 7f9f24ff9700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9f1406ace0 0x7f9f1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.441+0000 7f9f24ff9700 1 -- 172.21.15.79:0/525874457 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9f2008bb80 con 0x7f9f28159220 2023-06-23T18:36:19.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.441+0000 7f9f27fff700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9f1406ace0 0x7f9f1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.442+0000 7f9f27fff700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9f1406ace0 0x7f9f1406d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c007a50 tx=0x7f9f1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:19.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.446+0000 7f9f24ff9700 1 -- 172.21.15.79:0/525874457 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9f20056cc0 con 0x7f9f28159220 2023-06-23T18:36:19.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.634+0000 7f9f2e265700 1 -- 172.21.15.79:0/525874457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f2804e1d0 con 0x7f9f28159220 2023-06-23T18:36:19.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.634+0000 7f9f24ff9700 1 -- 172.21.15.79:0/525874457 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f2001f020 con 0x7f9f28159220 2023-06-23T18:36:19.640 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:19.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.636+0000 7f9f2e265700 1 -- 172.21.15.79:0/525874457 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9f1406ace0 msgr2=0x7f9f1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.636+0000 7f9f2e265700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9f1406ace0 0x7f9f1406d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c007a50 tx=0x7f9f1c008040 comp rx=0 tx=0).stop 2023-06-23T18:36:19.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.636+0000 7f9f2e265700 1 -- 172.21.15.79:0/525874457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f28159220 msgr2=0x7f9f280afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.636+0000 7f9f2e265700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f28159220 0x7f9f280afd00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9f200021c0 tx=0x7f9f200021f0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.636+0000 7f9f2e265700 1 -- 172.21.15.79:0/525874457 shutdown_connections 2023-06-23T18:36:19.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.636+0000 7f9f2e265700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f28159220 0x7f9f280afd00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.636+0000 7f9f2e265700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f280703a0 0x7f9f2814e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.636+0000 7f9f2e265700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9f1406ace0 0x7f9f1406d190 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.636+0000 7f9f2e265700 1 --2- 172.21.15.79:0/525874457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f280cc3d0 0x7f9f281e4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.636+0000 7f9f2e265700 1 -- 172.21.15.79:0/525874457 >> 172.21.15.79:0/525874457 conn(0x7f9f280cbf10 msgr2=0x7f9f281595f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:19.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.636+0000 7f9f2e265700 1 -- 172.21.15.79:0/525874457 shutdown_connections 2023-06-23T18:36:19.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.636+0000 7f9f2e265700 1 -- 172.21.15.79:0/525874457 wait complete. 2023-06-23T18:36:19.653 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:19.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.770+0000 7f50447c1700 1 Processor -- start 2023-06-23T18:36:19.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.770+0000 7f50447c1700 1 -- start start 2023-06-23T18:36:19.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.771+0000 7f50447c1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f503c06a5a0 0x7f503c1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.771+0000 7f50447c1700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c0cdc90 con 0x7f503c1569c0 2023-06-23T18:36:19.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.771+0000 7f50447c1700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c15d0f0 con 0x7f503c0cd440 2023-06-23T18:36:19.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.771+0000 7f50447c1700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503c0cf570 con 0x7f503c06a5a0 2023-06-23T18:36:19.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.771+0000 7f5041d5c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f503c06a5a0 0x7f503c1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.771+0000 7f504255d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f503c1569c0 0x7f503c069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43060/0 (socket says 172.21.15.79:43060) 2023-06-23T18:36:19.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.772+0000 7f504255d700 1 -- 172.21.15.79:0/2844701153 learned_addr learned my addr 172.21.15.79:0/2844701153 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:19.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.772+0000 7f5041d5c700 1 -- 172.21.15.79:0/2844701153 >> v1:172.21.15.155:6789/0 conn(0x7f503c0cd440 legacy=0x7f503c0b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.772+0000 7f5041d5c700 1 -- 172.21.15.79:0/2844701153 >> v1:172.21.15.79:6789/0 conn(0x7f503c1569c0 legacy=0x7f503c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.772+0000 7f5041d5c700 1 -- 172.21.15.79:0/2844701153 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c156d90 con 0x7f503c06a5a0 2023-06-23T18:36:19.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.772+0000 7f5041d5c700 1 --2- 172.21.15.79:0/2844701153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f503c06a5a0 0x7f503c1608b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5038009840 tx=0x7f5038009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7ad2c5e15f9f6b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:19.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.772+0000 7f504155b700 1 -- 172.21.15.79:0/2844701153 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5038004230 con 0x7f503c06a5a0 2023-06-23T18:36:19.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.772+0000 7f504155b700 1 -- 172.21.15.79:0/2844701153 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5038004390 con 0x7f503c06a5a0 2023-06-23T18:36:19.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.772+0000 7f504155b700 1 -- 172.21.15.79:0/2844701153 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50380036a0 con 0x7f503c06a5a0 2023-06-23T18:36:19.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.773+0000 7f50447c1700 1 -- 172.21.15.79:0/2844701153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f503c06a5a0 msgr2=0x7f503c1608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.773+0000 7f50447c1700 1 --2- 172.21.15.79:0/2844701153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f503c06a5a0 0x7f503c1608b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5038009840 tx=0x7f5038009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:19.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.773+0000 7f50447c1700 1 -- 172.21.15.79:0/2844701153 shutdown_connections 2023-06-23T18:36:19.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.773+0000 7f50447c1700 1 --2- 172.21.15.79:0/2844701153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f503c06a5a0 0x7f503c1608b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.773+0000 7f50447c1700 1 -- 172.21.15.79:0/2844701153 >> 172.21.15.79:0/2844701153 conn(0x7f503c06afa0 msgr2=0x7f503c06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:19.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.773+0000 7f50447c1700 1 -- 172.21.15.79:0/2844701153 shutdown_connections 2023-06-23T18:36:19.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.773+0000 7f50447c1700 1 -- 172.21.15.79:0/2844701153 wait complete. 2023-06-23T18:36:19.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.774+0000 7f50447c1700 1 Processor -- start 2023-06-23T18:36:19.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.774+0000 7f50447c1700 1 -- start start 2023-06-23T18:36:19.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f50447c1700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f503c06a5a0 0x7f503c0b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f50447c1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f503c0cd440 0x7f503c0c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f50447c1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f503c1569c0 0x7f503c1e8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f50447c1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c0c9820 con 0x7f503c0cd440 2023-06-23T18:36:19.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f50447c1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c1e7060 con 0x7f503c1569c0 2023-06-23T18:36:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f50447c1700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503c0b18b0 con 0x7f503c06a5a0 2023-06-23T18:36:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f5041d5c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f503c0cd440 0x7f503c0c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f5041d5c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f503c0cd440 0x7f503c0c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40024/0 (socket says 172.21.15.79:40024) 2023-06-23T18:36:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f5041d5c700 1 -- 172.21.15.79:0/4138346340 learned_addr learned my addr 172.21.15.79:0/4138346340 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f5042d5e700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f503c1569c0 0x7f503c1e8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f504255d700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f503c06a5a0 0x7f503c0b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f5041d5c700 1 -- 172.21.15.79:0/4138346340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f503c06a5a0 msgr2=0x7f503c0b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f5041d5c700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f503c06a5a0 0x7f503c0b0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f5041d5c700 1 -- 172.21.15.79:0/4138346340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f503c1569c0 msgr2=0x7f503c1e8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f5041d5c700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f503c1569c0 0x7f503c1e8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f5041d5c700 1 -- 172.21.15.79:0/4138346340 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50380094a0 con 0x7f503c0cd440 2023-06-23T18:36:19.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f5042d5e700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f503c1569c0 0x7f503c1e8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:19.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.775+0000 7f5041d5c700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f503c0cd440 0x7f503c0c92e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f50380090b0 tx=0x7f5038004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:19.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.776+0000 7f5032ffd700 1 -- 172.21.15.79:0/4138346340 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5038011830 con 0x7f503c0cd440 2023-06-23T18:36:19.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.776+0000 7f5032ffd700 1 -- 172.21.15.79:0/4138346340 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503801f070 con 0x7f503c0cd440 2023-06-23T18:36:19.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.776+0000 7f50447c1700 1 -- 172.21.15.79:0/4138346340 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503c155bd0 con 0x7f503c0cd440 2023-06-23T18:36:19.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.776+0000 7f5032ffd700 1 -- 172.21.15.79:0/4138346340 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5038011bd0 con 0x7f503c0cd440 2023-06-23T18:36:19.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.776+0000 7f50447c1700 1 -- 172.21.15.79:0/4138346340 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503c1e7540 con 0x7f503c0cd440 2023-06-23T18:36:19.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.776+0000 7f50447c1700 1 -- 172.21.15.79:0/4138346340 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503c0672c0 con 0x7f503c0cd440 2023-06-23T18:36:19.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.777+0000 7f5032ffd700 1 -- 172.21.15.79:0/4138346340 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5038011d30 con 0x7f503c0cd440 2023-06-23T18:36:19.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.777+0000 7f5032ffd700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f502006ad10 0x7f502006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:19.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.777+0000 7f5032ffd700 1 -- 172.21.15.79:0/4138346340 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5038021070 con 0x7f503c0cd440 2023-06-23T18:36:19.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.777+0000 7f504255d700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f502006ad10 0x7f502006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:19.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.778+0000 7f504255d700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f502006ad10 0x7f502006d1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f502c000f60 tx=0x7f502c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:19.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.782+0000 7f5032ffd700 1 -- 172.21.15.79:0/4138346340 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f50380560e0 con 0x7f503c0cd440 2023-06-23T18:36:19.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.969+0000 7f50447c1700 1 -- 172.21.15.79:0/4138346340 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f503c155e20 con 0x7f503c0cd440 2023-06-23T18:36:19.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.970+0000 7f5032ffd700 1 -- 172.21.15.79:0/4138346340 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5038059700 con 0x7f503c0cd440 2023-06-23T18:36:19.975 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:19.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.972+0000 7f50447c1700 1 -- 172.21.15.79:0/4138346340 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f502006ad10 msgr2=0x7f502006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.972+0000 7f50447c1700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f502006ad10 0x7f502006d1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f502c000f60 tx=0x7f502c009040 comp rx=0 tx=0).stop 2023-06-23T18:36:19.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.972+0000 7f50447c1700 1 -- 172.21.15.79:0/4138346340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f503c0cd440 msgr2=0x7f503c0c92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:19.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.972+0000 7f50447c1700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f503c0cd440 0x7f503c0c92e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f50380090b0 tx=0x7f5038004750 comp rx=0 tx=0).stop 2023-06-23T18:36:19.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.972+0000 7f50447c1700 1 -- 172.21.15.79:0/4138346340 shutdown_connections 2023-06-23T18:36:19.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.972+0000 7f50447c1700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f502006ad10 0x7f502006d1c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.972+0000 7f50447c1700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f503c1569c0 0x7f503c1e8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.972+0000 7f50447c1700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f503c06a5a0 0x7f503c0b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.972+0000 7f50447c1700 1 --2- 172.21.15.79:0/4138346340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f503c0cd440 0x7f503c0c92e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:19.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.972+0000 7f50447c1700 1 -- 172.21.15.79:0/4138346340 >> 172.21.15.79:0/4138346340 conn(0x7f503c06afa0 msgr2=0x7f503c15c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:19.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.972+0000 7f50447c1700 1 -- 172.21.15.79:0/4138346340 shutdown_connections 2023-06-23T18:36:19.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:19.972+0000 7f50447c1700 1 -- 172.21.15.79:0/4138346340 wait complete. 2023-06-23T18:36:19.989 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:20.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.104+0000 7fcfa3ecc700 1 Processor -- start 2023-06-23T18:36:20.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.105+0000 7fcfa3ecc700 1 -- start start 2023-06-23T18:36:20.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.105+0000 7fcfa3ecc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcf9c15a380 0x7fcf9c0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.105+0000 7fcfa3ecc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf9c15a750 0x7fcf9c0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.105+0000 7fcfa3ecc700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcf9c0ade50 0x7fcf9c0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.105+0000 7fcfa3ecc700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf9c0b1d40 con 0x7fcf9c15a380 2023-06-23T18:36:20.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.105+0000 7fcfa3ecc700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf9c0b2020 con 0x7fcf9c0ade50 2023-06-23T18:36:20.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.105+0000 7fcfa3ecc700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c0ad650 con 0x7fcf9c15a750 2023-06-23T18:36:20.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.105+0000 7fcfa1c68700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcf9c15a380 0x7fcf9c0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.106+0000 7fcfa1c68700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcf9c15a380 0x7fcf9c0ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40030/0 (socket says 172.21.15.79:40030) 2023-06-23T18:36:20.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.106+0000 7fcfa1c68700 1 -- 172.21.15.79:0/2139924170 learned_addr learned my addr 172.21.15.79:0/2139924170 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:20.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.106+0000 7fcfa1467700 1 --2- 172.21.15.79:0/2139924170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf9c15a750 0x7fcf9c0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.106+0000 7fcfa2469700 1 --2- 172.21.15.79:0/2139924170 >> v2:172.21.15.155:3300/0 conn(0x7fcf9c0ade50 0x7fcf9c0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.106+0000 7fcfa1c68700 1 -- 172.21.15.79:0/2139924170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf9c15a750 msgr2=0x7fcf9c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:20.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.106+0000 7fcfa1c68700 1 --2- 172.21.15.79:0/2139924170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf9c15a750 0x7fcf9c0cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.106+0000 7fcfa1c68700 1 -- 172.21.15.79:0/2139924170 >> v2:172.21.15.155:3300/0 conn(0x7fcf9c0ade50 msgr2=0x7fcf9c0ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:20.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.106+0000 7fcfa1c68700 1 --2- 172.21.15.79:0/2139924170 >> v2:172.21.15.155:3300/0 conn(0x7fcf9c0ade50 0x7fcf9c0ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.106+0000 7fcfa1c68700 1 -- 172.21.15.79:0/2139924170 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c0ad930 con 0x7fcf9c15a380 2023-06-23T18:36:20.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.106+0000 7fcfa1c68700 1 --2- 172.21.15.79:0/2139924170 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c15a380 0x7fcf9c0ce4f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0cff50 tx=0x7fcf8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a921f975c67bf724 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:20.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.106+0000 7fcfa0c66700 1 -- 172.21.15.79:0/2139924170 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c005e30 con 0x7fcf9c15a380 2023-06-23T18:36:20.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.106+0000 7fcfa0c66700 1 -- 172.21.15.79:0/2139924170 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c004030 con 0x7fcf9c15a380 2023-06-23T18:36:20.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.106+0000 7fcfa0c66700 1 -- 172.21.15.79:0/2139924170 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c0043e0 con 0x7fcf9c15a380 2023-06-23T18:36:20.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.107+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/2139924170 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c15a380 msgr2=0x7fcf9c0ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.107+0000 7fcfa3ecc700 1 --2- 172.21.15.79:0/2139924170 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c15a380 0x7fcf9c0ce4f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0cff50 tx=0x7fcf8c009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.107+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/2139924170 shutdown_connections 2023-06-23T18:36:20.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.107+0000 7fcfa3ecc700 1 --2- 172.21.15.79:0/2139924170 >> v2:172.21.15.155:3300/0 conn(0x7fcf9c0ade50 0x7fcf9c0ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.107+0000 7fcfa3ecc700 1 --2- 172.21.15.79:0/2139924170 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c15a380 0x7fcf9c0ce4f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.107+0000 7fcfa3ecc700 1 --2- 172.21.15.79:0/2139924170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf9c15a750 0x7fcf9c0cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.107+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/2139924170 >> 172.21.15.79:0/2139924170 conn(0x7fcf9c089fc0 msgr2=0x7fcf9c099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:20.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.107+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/2139924170 shutdown_connections 2023-06-23T18:36:20.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.107+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/2139924170 wait complete. 2023-06-23T18:36:20.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.108+0000 7fcfa3ecc700 1 Processor -- start 2023-06-23T18:36:20.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.108+0000 7fcfa3ecc700 1 -- start start 2023-06-23T18:36:20.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.109+0000 7fcfa3ecc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c0ade50 0x7fcf9c15d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.109+0000 7fcfa3ecc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf9c15a380 0x7fcf9c0c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.109+0000 7fcfa3ecc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf9c15a750 0x7fcf9c0c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.109+0000 7fcfa3ecc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c147750 con 0x7fcf9c0ade50 2023-06-23T18:36:20.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.109+0000 7fcfa3ecc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c0b3750 con 0x7fcf9c15a380 2023-06-23T18:36:20.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.109+0000 7fcfa3ecc700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c15cc80 con 0x7fcf9c15a750 2023-06-23T18:36:20.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.109+0000 7fcfa1c68700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c0ade50 0x7fcf9c15d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.109+0000 7fcfa1c68700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c0ade50 0x7fcf9c15d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40036/0 (socket says 172.21.15.79:40036) 2023-06-23T18:36:20.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.109+0000 7fcfa1c68700 1 -- 172.21.15.79:0/3605567262 learned_addr learned my addr 172.21.15.79:0/3605567262 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:20.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.109+0000 7fcfa1467700 1 --2- 172.21.15.79:0/3605567262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf9c15a380 0x7fcf9c0c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.109+0000 7fcfa2469700 1 --2- 172.21.15.79:0/3605567262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf9c15a750 0x7fcf9c0c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.110+0000 7fcfa1c68700 1 -- 172.21.15.79:0/3605567262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf9c15a750 msgr2=0x7fcf9c0c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.110+0000 7fcfa1c68700 1 --2- 172.21.15.79:0/3605567262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf9c15a750 0x7fcf9c0c4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.110+0000 7fcfa1c68700 1 -- 172.21.15.79:0/3605567262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf9c15a380 msgr2=0x7fcf9c0c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.110+0000 7fcfa1c68700 1 --2- 172.21.15.79:0/3605567262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf9c15a380 0x7fcf9c0c45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.110+0000 7fcfa1c68700 1 -- 172.21.15.79:0/3605567262 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c009500 con 0x7fcf9c0ade50 2023-06-23T18:36:20.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.110+0000 7fcfa1c68700 1 --2- 172.21.15.79:0/3605567262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c0ade50 0x7fcf9c15d110 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c06d340 tx=0x7fcf8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:20.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.110+0000 7fcf927fc700 1 -- 172.21.15.79:0/3605567262 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c003bb0 con 0x7fcf9c0ade50 2023-06-23T18:36:20.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.110+0000 7fcf927fc700 1 -- 172.21.15.79:0/3605567262 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c003d10 con 0x7fcf9c0ade50 2023-06-23T18:36:20.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.110+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/3605567262 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9c15c670 con 0x7fcf9c0ade50 2023-06-23T18:36:20.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.110+0000 7fcf927fc700 1 -- 172.21.15.79:0/3605567262 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c01c660 con 0x7fcf9c0ade50 2023-06-23T18:36:20.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.110+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/3605567262 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf9c15c910 con 0x7fcf9c0ade50 2023-06-23T18:36:20.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.111+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/3605567262 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9c15cf60 con 0x7fcf9c0ade50 2023-06-23T18:36:20.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.111+0000 7fcf927fc700 1 -- 172.21.15.79:0/3605567262 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcf8c01c7c0 con 0x7fcf9c0ade50 2023-06-23T18:36:20.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.112+0000 7fcf927fc700 1 --2- 172.21.15.79:0/3605567262 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcf8006ace0 0x7fcf8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.112+0000 7fcf927fc700 1 -- 172.21.15.79:0/3605567262 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcf8c08a280 con 0x7fcf9c0ade50 2023-06-23T18:36:20.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.112+0000 7fcfa1467700 1 --2- 172.21.15.79:0/3605567262 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcf8006ace0 0x7fcf8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.112+0000 7fcfa1467700 1 --2- 172.21.15.79:0/3605567262 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcf8006ace0 0x7fcf8006d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0b0e00 tx=0x7fcf98009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:20.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.116+0000 7fcf927fc700 1 -- 172.21.15.79:0/3605567262 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcf8c0553c0 con 0x7fcf9c0ade50 2023-06-23T18:36:20.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.306+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/3605567262 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf9c15cad0 con 0x7fcf9c0ade50 2023-06-23T18:36:20.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.307+0000 7fcf927fc700 1 -- 172.21.15.79:0/3605567262 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7fcf8c0589e0 con 0x7fcf9c0ade50 2023-06-23T18:36:20.312 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:20.313 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:20.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.309+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/3605567262 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcf8006ace0 msgr2=0x7fcf8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.309+0000 7fcfa3ecc700 1 --2- 172.21.15.79:0/3605567262 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcf8006ace0 0x7fcf8006d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0b0e00 tx=0x7fcf98009210 comp rx=0 tx=0).stop 2023-06-23T18:36:20.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.309+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/3605567262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c0ade50 msgr2=0x7fcf9c15d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.309+0000 7fcfa3ecc700 1 --2- 172.21.15.79:0/3605567262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c0ade50 0x7fcf9c15d110 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c06d340 tx=0x7fcf8c004750 comp rx=0 tx=0).stop 2023-06-23T18:36:20.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.309+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/3605567262 shutdown_connections 2023-06-23T18:36:20.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.309+0000 7fcfa3ecc700 1 --2- 172.21.15.79:0/3605567262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf9c15a750 0x7fcf9c0c4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.309+0000 7fcfa3ecc700 1 --2- 172.21.15.79:0/3605567262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c0ade50 0x7fcf9c15d110 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.309+0000 7fcfa3ecc700 1 --2- 172.21.15.79:0/3605567262 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcf8006ace0 0x7fcf8006d190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.309+0000 7fcfa3ecc700 1 --2- 172.21.15.79:0/3605567262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf9c15a380 0x7fcf9c0c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.309+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/3605567262 >> 172.21.15.79:0/3605567262 conn(0x7fcf9c089fc0 msgr2=0x7fcf9c0aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:20.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.309+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/3605567262 shutdown_connections 2023-06-23T18:36:20.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.309+0000 7fcfa3ecc700 1 -- 172.21.15.79:0/3605567262 wait complete. 2023-06-23T18:36:20.326 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:20.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:20.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.445+0000 7fb0a661c700 1 Processor -- start 2023-06-23T18:36:20.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.445+0000 7fb0a661c700 1 -- start start 2023-06-23T18:36:20.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.445+0000 7fb0a661c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00cedf0 0x7fb0a00b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.445+0000 7fb0a661c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a00b22c0 con 0x7fb0a01438f0 2023-06-23T18:36:20.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.445+0000 7fb0a661c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a00b0b10 con 0x7fb0a00d08c0 2023-06-23T18:36:20.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.445+0000 7fb0a661c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00b0df0 con 0x7fb0a00cedf0 2023-06-23T18:36:20.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09f7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fb0a01438f0 0x7fb0a00b1bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43086/0 (socket says 172.21.15.79:43086) 2023-06-23T18:36:20.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09f7fe700 1 -- 172.21.15.79:0/2188176612 learned_addr learned my addr 172.21.15.79:0/2188176612 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:20.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09ffff700 1 --2- 172.21.15.79:0/2188176612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00cedf0 0x7fb0a00b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 -- 172.21.15.79:0/2188176612 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646599544 0 0) 0x7fb0a00b22c0 con 0x7fb0a01438f0 2023-06-23T18:36:20.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 -- 172.21.15.79:0/2188176612 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0880035b0 con 0x7fb0a01438f0 2023-06-23T18:36:20.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 -- 172.21.15.79:0/2188176612 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1680259648 0 0) 0x7fb0880035b0 con 0x7fb0a01438f0 2023-06-23T18:36:20.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 -- 172.21.15.79:0/2188176612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00cedf0 msgr2=0x7fb0a00b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:20.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 --2- 172.21.15.79:0/2188176612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00cedf0 0x7fb0a00b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 -- 172.21.15.79:0/2188176612 >> v1:172.21.15.155:6789/0 conn(0x7fb0a00d08c0 legacy=0x7fb0a00ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 -- 172.21.15.79:0/2188176612 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a00b10d0 con 0x7fb0a01438f0 2023-06-23T18:36:20.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 -- 172.21.15.79:0/2188176612 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fb094001670 con 0x7fb0a01438f0 2023-06-23T18:36:20.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 -- 172.21.15.79:0/2188176612 >> v1:172.21.15.79:6789/0 conn(0x7fb0a01438f0 legacy=0x7fb0a00b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 --2- 172.21.15.79:0/2188176612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb088003a30 0x7fb088003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 --2- 172.21.15.79:0/2188176612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb088004850 0x7fb088004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 --2- 172.21.15.79:0/2188176612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00cedf0 0x7fb088005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 -- 172.21.15.79:0/2188176612 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00b0df0 con 0x7fb088003a30 2023-06-23T18:36:20.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 -- 172.21.15.79:0/2188176612 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08800bb10 con 0x7fb088004850 2023-06-23T18:36:20.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09effd700 1 -- 172.21.15.79:0/2188176612 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08800bcd0 con 0x7fb0a00cedf0 2023-06-23T18:36:20.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09ffff700 1 --2- 172.21.15.79:0/2188176612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb088003a30 0x7fb088003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09f7fe700 1 --2- 172.21.15.79:0/2188176612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb088004850 0x7fb088004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb0a4bb9700 1 --2- 172.21.15.79:0/2188176612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00cedf0 0x7fb088005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09ffff700 1 -- 172.21.15.79:0/2188176612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00cedf0 msgr2=0x7fb088005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:20.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09ffff700 1 --2- 172.21.15.79:0/2188176612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00cedf0 0x7fb088005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09ffff700 1 -- 172.21.15.79:0/2188176612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb088004850 msgr2=0x7fb088004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:20.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09ffff700 1 --2- 172.21.15.79:0/2188176612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb088004850 0x7fb088004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.446+0000 7fb09ffff700 1 -- 172.21.15.79:0/2188176612 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0880035b0 con 0x7fb088003a30 2023-06-23T18:36:20.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.447+0000 7fb09ffff700 1 --2- 172.21.15.79:0/2188176612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb088003a30 0x7fb088003e40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb090009b40 tx=0x7fb088005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c64dd68f2ab04c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:20.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.447+0000 7fb09effd700 1 -- 172.21.15.79:0/2188176612 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090005a20 con 0x7fb088003a30 2023-06-23T18:36:20.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.447+0000 7fb09effd700 1 -- 172.21.15.79:0/2188176612 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb090005b80 con 0x7fb088003a30 2023-06-23T18:36:20.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.447+0000 7fb09effd700 1 -- 172.21.15.79:0/2188176612 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09000e6c0 con 0x7fb088003a30 2023-06-23T18:36:20.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.447+0000 7fb0a661c700 1 -- 172.21.15.79:0/2188176612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb088003a30 msgr2=0x7fb088003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.447+0000 7fb0a661c700 1 --2- 172.21.15.79:0/2188176612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb088003a30 0x7fb088003e40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb090009b40 tx=0x7fb088005880 comp rx=0 tx=0).stop 2023-06-23T18:36:20.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.447+0000 7fb0a4bb9700 1 -- 172.21.15.79:0/2188176612 reap_dead start 2023-06-23T18:36:20.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.447+0000 7fb0a661c700 1 -- 172.21.15.79:0/2188176612 shutdown_connections 2023-06-23T18:36:20.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.447+0000 7fb0a661c700 1 -- 172.21.15.79:0/2188176612 >> 172.21.15.79:0/2188176612 conn(0x7fb0a006a7b0 msgr2=0x7fb0a0099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:20.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.447+0000 7fb0a661c700 1 -- 172.21.15.79:0/2188176612 shutdown_connections 2023-06-23T18:36:20.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.447+0000 7fb0a661c700 1 -- 172.21.15.79:0/2188176612 wait complete. 2023-06-23T18:36:20.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.449+0000 7fb0a661c700 1 Processor -- start 2023-06-23T18:36:20.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.449+0000 7fb0a661c700 1 -- start start 2023-06-23T18:36:20.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.449+0000 7fb0a661c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb088003a30 0x7fb0a00cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.449+0000 7fb0a661c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a015d320 0x7fb0a00c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.449+0000 7fb0a661c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a01547c0 0x7fb0a0154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.449+0000 7fb0a661c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a0155180 con 0x7fb088003a30 2023-06-23T18:36:20.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.449+0000 7fb0a661c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00cd640 con 0x7fb0a01547c0 2023-06-23T18:36:20.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.449+0000 7fb0a661c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00cd080 con 0x7fb0a015d320 2023-06-23T18:36:20.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.450+0000 7fb09ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb088003a30 0x7fb0a00cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.450+0000 7fb09ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb088003a30 0x7fb0a00cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40056/0 (socket says 172.21.15.79:40056) 2023-06-23T18:36:20.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.450+0000 7fb09ffff700 1 -- 172.21.15.79:0/3426989581 learned_addr learned my addr 172.21.15.79:0/3426989581 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:20.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.450+0000 7fb09ffff700 1 -- 172.21.15.79:0/3426989581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a015d320 msgr2=0x7fb0a00c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.450+0000 7fb09ffff700 1 --2- 172.21.15.79:0/3426989581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a015d320 0x7fb0a00c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.450+0000 7fb09ffff700 1 -- 172.21.15.79:0/3426989581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a01547c0 msgr2=0x7fb0a0154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.450+0000 7fb09ffff700 1 --2- 172.21.15.79:0/3426989581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a01547c0 0x7fb0a0154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.450+0000 7fb09ffff700 1 -- 172.21.15.79:0/3426989581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a00cd360 con 0x7fb088003a30 2023-06-23T18:36:20.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.450+0000 7fb09ffff700 1 --2- 172.21.15.79:0/3426989581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb088003a30 0x7fb0a00cc850 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb08800bae0 tx=0x7fb0900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:20.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.450+0000 7fb09cff9700 1 -- 172.21.15.79:0/3426989581 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090020070 con 0x7fb088003a30 2023-06-23T18:36:20.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.450+0000 7fb09cff9700 1 -- 172.21.15.79:0/3426989581 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb090004030 con 0x7fb088003a30 2023-06-23T18:36:20.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.450+0000 7fb0a661c700 1 -- 172.21.15.79:0/3426989581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a006b0d0 con 0x7fb088003a30 2023-06-23T18:36:20.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.450+0000 7fb09cff9700 1 -- 172.21.15.79:0/3426989581 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090004440 con 0x7fb088003a30 2023-06-23T18:36:20.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.450+0000 7fb0a661c700 1 -- 172.21.15.79:0/3426989581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a00ccd90 con 0x7fb088003a30 2023-06-23T18:36:20.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.451+0000 7fb0a661c700 1 -- 172.21.15.79:0/3426989581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a0155180 con 0x7fb088003a30 2023-06-23T18:36:20.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.451+0000 7fb09cff9700 1 -- 172.21.15.79:0/3426989581 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb0900036a0 con 0x7fb088003a30 2023-06-23T18:36:20.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.452+0000 7fb09cff9700 1 --2- 172.21.15.79:0/3426989581 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb08c06ace0 0x7fb08c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.452+0000 7fb09cff9700 1 -- 172.21.15.79:0/3426989581 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb09008a710 con 0x7fb088003a30 2023-06-23T18:36:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.452+0000 7fb09f7fe700 1 --2- 172.21.15.79:0/3426989581 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb08c06ace0 0x7fb08c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.452+0000 7fb09f7fe700 1 --2- 172.21.15.79:0/3426989581 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb08c06ace0 0x7fb08c06d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb094001eb0 tx=0x7fb094001d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.456+0000 7fb09cff9700 1 -- 172.21.15.79:0/3426989581 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb0a0155180 con 0x7fb088003a30 2023-06-23T18:36:20.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.644+0000 7fb0a661c700 1 -- 172.21.15.79:0/3426989581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb0a0155180 con 0x7fb088003a30 2023-06-23T18:36:20.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.644+0000 7fb09cff9700 1 -- 172.21.15.79:0/3426989581 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb0a0155180 con 0x7fb088003a30 2023-06-23T18:36:20.649 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:20.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.646+0000 7fb0a661c700 1 -- 172.21.15.79:0/3426989581 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb08c06ace0 msgr2=0x7fb08c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.646+0000 7fb0a661c700 1 --2- 172.21.15.79:0/3426989581 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb08c06ace0 0x7fb08c06d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb094001eb0 tx=0x7fb094001d60 comp rx=0 tx=0).stop 2023-06-23T18:36:20.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.646+0000 7fb0a661c700 1 -- 172.21.15.79:0/3426989581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb088003a30 msgr2=0x7fb0a00cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.646+0000 7fb0a661c700 1 --2- 172.21.15.79:0/3426989581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb088003a30 0x7fb0a00cc850 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb08800bae0 tx=0x7fb0900047c0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.646+0000 7fb0a661c700 1 -- 172.21.15.79:0/3426989581 shutdown_connections 2023-06-23T18:36:20.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.646+0000 7fb0a661c700 1 --2- 172.21.15.79:0/3426989581 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb08c06ace0 0x7fb08c06d190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.646+0000 7fb0a661c700 1 --2- 172.21.15.79:0/3426989581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a01547c0 0x7fb0a0154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.646+0000 7fb0a661c700 1 --2- 172.21.15.79:0/3426989581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb088003a30 0x7fb0a00cc850 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.646+0000 7fb0a661c700 1 --2- 172.21.15.79:0/3426989581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a015d320 0x7fb0a00c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.646+0000 7fb0a661c700 1 -- 172.21.15.79:0/3426989581 >> 172.21.15.79:0/3426989581 conn(0x7fb0a006a7b0 msgr2=0x7fb0a006f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:20.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.646+0000 7fb0a661c700 1 -- 172.21.15.79:0/3426989581 shutdown_connections 2023-06-23T18:36:20.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.646+0000 7fb0a661c700 1 -- 172.21.15.79:0/3426989581 wait complete. 2023-06-23T18:36:20.663 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:20.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.782+0000 7f72d7bc4700 1 Processor -- start 2023-06-23T18:36:20.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.782+0000 7f72d7bc4700 1 -- start start 2023-06-23T18:36:20.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.783+0000 7f72d7bc4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d0156ae0 0x7f72d015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.783+0000 7f72d7bc4700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d015daf0 con 0x7f72d00cc060 2023-06-23T18:36:20.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.783+0000 7f72d7bc4700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d015a560 con 0x7f72d00cc4d0 2023-06-23T18:36:20.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.783+0000 7f72d7bc4700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d0072530 con 0x7f72d0156ae0 2023-06-23T18:36:20.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.783+0000 7f72d515f700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f72d00cc060 0x7f72d0071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43106/0 (socket says 172.21.15.79:43106) 2023-06-23T18:36:20.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.783+0000 7f72d515f700 1 -- 172.21.15.79:0/2348438843 learned_addr learned my addr 172.21.15.79:0/2348438843 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:20.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.783+0000 7f72d5960700 1 --2- 172.21.15.79:0/2348438843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d0156ae0 0x7f72d015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.783+0000 7f72d495e700 1 -- 172.21.15.79:0/2348438843 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769974613 0 0) 0x7f72d015daf0 con 0x7f72d00cc060 2023-06-23T18:36:20.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.783+0000 7f72d495e700 1 -- 172.21.15.79:0/2348438843 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72bc0035b0 con 0x7f72d00cc060 2023-06-23T18:36:20.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.783+0000 7f72d5960700 1 -- 172.21.15.79:0/2348438843 >> v1:172.21.15.155:6789/0 conn(0x7f72d00cc4d0 legacy=0x7f72d0071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.783+0000 7f72d495e700 1 -- 172.21.15.79:0/2348438843 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214090087 0 0) 0x7f72d015a560 con 0x7f72d00cc4d0 2023-06-23T18:36:20.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.783+0000 7f72d5960700 1 -- 172.21.15.79:0/2348438843 >> v1:172.21.15.79:6789/0 conn(0x7f72d00cc060 legacy=0x7f72d0071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.783+0000 7f72d5960700 1 -- 172.21.15.79:0/2348438843 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d0159950 con 0x7f72d0156ae0 2023-06-23T18:36:20.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.784+0000 7f72d5960700 1 --2- 172.21.15.79:0/2348438843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d0156ae0 0x7f72d015d5b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009870 tx=0x7f72cc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f545d851a4b3b4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:20.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.784+0000 7f72d495e700 1 -- 172.21.15.79:0/2348438843 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc011830 con 0x7f72d0156ae0 2023-06-23T18:36:20.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.784+0000 7f72d495e700 1 -- 172.21.15.79:0/2348438843 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc011990 con 0x7f72d0156ae0 2023-06-23T18:36:20.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.784+0000 7f72d495e700 1 -- 172.21.15.79:0/2348438843 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc011d20 con 0x7f72d0156ae0 2023-06-23T18:36:20.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.784+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2348438843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d0156ae0 msgr2=0x7f72d015d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.784+0000 7f72d7bc4700 1 --2- 172.21.15.79:0/2348438843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d0156ae0 0x7f72d015d5b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009870 tx=0x7f72cc009b80 comp rx=0 tx=0).stop 2023-06-23T18:36:20.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.784+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2348438843 shutdown_connections 2023-06-23T18:36:20.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.784+0000 7f72d7bc4700 1 --2- 172.21.15.79:0/2348438843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d0156ae0 0x7f72d015d5b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.784+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2348438843 >> 172.21.15.79:0/2348438843 conn(0x7f72d006afa0 msgr2=0x7f72d006b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:20.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.784+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2348438843 shutdown_connections 2023-06-23T18:36:20.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.784+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2348438843 wait complete. 2023-06-23T18:36:20.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.786+0000 7f72d7bc4700 1 Processor -- start 2023-06-23T18:36:20.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.786+0000 7f72d7bc4700 1 -- start start 2023-06-23T18:36:20.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.786+0000 7f72d7bc4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d00cc060 0x7f72d00b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.786+0000 7f72d7bc4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72d00cc4d0 0x7f72d00b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.786+0000 7f72d7bc4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d0156ae0 0x7f72d0142b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.786+0000 7f72d7bc4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00c4e10 con 0x7f72d00cc4d0 2023-06-23T18:36:20.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.786+0000 7f72d7bc4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00c50b0 con 0x7f72d00cc060 2023-06-23T18:36:20.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.786+0000 7f72d7bc4700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d01f40f0 con 0x7f72d0156ae0 2023-06-23T18:36:20.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.786+0000 7f72d515f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72d00cc4d0 0x7f72d00b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.786+0000 7f72d515f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72d00cc4d0 0x7f72d00b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40070/0 (socket says 172.21.15.79:40070) 2023-06-23T18:36:20.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.786+0000 7f72d515f700 1 -- 172.21.15.79:0/2762235682 learned_addr learned my addr 172.21.15.79:0/2762235682 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:20.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.787+0000 7f72d5960700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d00cc060 0x7f72d00b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.787+0000 7f72d6161700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d0156ae0 0x7f72d0142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.787+0000 7f72d515f700 1 -- 172.21.15.79:0/2762235682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d0156ae0 msgr2=0x7f72d0142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.787+0000 7f72d515f700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d0156ae0 0x7f72d0142b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.787+0000 7f72d515f700 1 -- 172.21.15.79:0/2762235682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d00cc060 msgr2=0x7f72d00b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.787+0000 7f72d515f700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d00cc060 0x7f72d00b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.787+0000 7f72d515f700 1 -- 172.21.15.79:0/2762235682 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc0094a0 con 0x7f72d00cc4d0 2023-06-23T18:36:20.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.787+0000 7f72d5960700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d00cc060 0x7f72d00b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:20.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.787+0000 7f72d515f700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72d00cc4d0 0x7f72d00b5160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f72c000c450 tx=0x7f72c000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:20.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.787+0000 7f72c67fc700 1 -- 172.21.15.79:0/2762235682 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0005d70 con 0x7f72d00cc4d0 2023-06-23T18:36:20.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.787+0000 7f72c67fc700 1 -- 172.21.15.79:0/2762235682 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c00070d0 con 0x7f72d00cc4d0 2023-06-23T18:36:20.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.787+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2762235682 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d00c3360 con 0x7f72d00cc4d0 2023-06-23T18:36:20.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.787+0000 7f72c67fc700 1 -- 172.21.15.79:0/2762235682 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0007440 con 0x7f72d00cc4d0 2023-06-23T18:36:20.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.787+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2762235682 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d00c3920 con 0x7f72d00cc4d0 2023-06-23T18:36:20.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.788+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2762235682 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d00672c0 con 0x7f72d00cc4d0 2023-06-23T18:36:20.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.788+0000 7f72c67fc700 1 -- 172.21.15.79:0/2762235682 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f72c00075a0 con 0x7f72d00cc4d0 2023-06-23T18:36:20.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.789+0000 7f72c67fc700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f72b406ace0 0x7f72b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:20.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.789+0000 7f72c67fc700 1 -- 172.21.15.79:0/2762235682 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f72c00887f0 con 0x7f72d00cc4d0 2023-06-23T18:36:20.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.789+0000 7f72d5960700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f72b406ace0 0x7f72b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:20.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.789+0000 7f72d5960700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f72b406ace0 0x7f72b406d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0097c0 tx=0x7f72cc01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:20.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.793+0000 7f72c67fc700 1 -- 172.21.15.79:0/2762235682 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f72c0053950 con 0x7f72d00cc4d0 2023-06-23T18:36:20.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.981+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2762235682 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72d00c51f0 con 0x7f72d00cc4d0 2023-06-23T18:36:20.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.981+0000 7f72c67fc700 1 -- 172.21.15.79:0/2762235682 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72c0017020 con 0x7f72d00cc4d0 2023-06-23T18:36:20.986 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:20.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.983+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2762235682 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f72b406ace0 msgr2=0x7f72b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.983+0000 7f72d7bc4700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f72b406ace0 0x7f72b406d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0097c0 tx=0x7f72cc01c040 comp rx=0 tx=0).stop 2023-06-23T18:36:20.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.983+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2762235682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72d00cc4d0 msgr2=0x7f72d00b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:20.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.983+0000 7f72d7bc4700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72d00cc4d0 0x7f72d00b5160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f72c000c450 tx=0x7f72c000c810 comp rx=0 tx=0).stop 2023-06-23T18:36:20.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.983+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2762235682 shutdown_connections 2023-06-23T18:36:20.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.983+0000 7f72d7bc4700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d0156ae0 0x7f72d0142b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.983+0000 7f72d7bc4700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f72b406ace0 0x7f72b406d190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.983+0000 7f72d7bc4700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d00cc060 0x7f72d00b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.983+0000 7f72d7bc4700 1 --2- 172.21.15.79:0/2762235682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72d00cc4d0 0x7f72d00b5160 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:20.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.983+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2762235682 >> 172.21.15.79:0/2762235682 conn(0x7f72d006afa0 msgr2=0x7f72d01590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:20.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.983+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2762235682 shutdown_connections 2023-06-23T18:36:20.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:20.983+0000 7f72d7bc4700 1 -- 172.21.15.79:0/2762235682 wait complete. 2023-06-23T18:36:21.000 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:21.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.118+0000 7f3afd7cb700 1 Processor -- start 2023-06-23T18:36:21.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.118+0000 7f3afd7cb700 1 -- start start 2023-06-23T18:36:21.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.119+0000 7f3afd7cb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3af81607a0 0x7f3af806a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.119+0000 7f3afd7cb700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af8160c20 0x7f3af806a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.119+0000 7f3afd7cb700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af8160030 con 0x7f3af815a290 2023-06-23T18:36:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.119+0000 7f3afd7cb700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af8160310 con 0x7f3af81607a0 2023-06-23T18:36:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.119+0000 7f3afd7cb700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80cd2d0 con 0x7f3af8160c20 2023-06-23T18:36:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.119+0000 7f3af67fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3af81607a0 0x7f3af806a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.119+0000 7f3af6ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3af815a290 0x7f3af815d470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43118/0 (socket says 172.21.15.79:43118) 2023-06-23T18:36:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.119+0000 7f3af67fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3af81607a0 0x7f3af806a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40078/0 (socket says 172.21.15.79:40078) 2023-06-23T18:36:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.119+0000 7f3af6ffd700 1 -- 172.21.15.79:0/1856741320 learned_addr learned my addr 172.21.15.79:0/1856741320 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:21.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.119+0000 7f3af77fe700 1 --2- 172.21.15.79:0/1856741320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af8160c20 0x7f3af806a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3af67fc700 1 -- 172.21.15.79:0/1856741320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af8160c20 msgr2=0x7f3af806a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:21.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3af67fc700 1 --2- 172.21.15.79:0/1856741320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af8160c20 0x7f3af806a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3af67fc700 1 -- 172.21.15.79:0/1856741320 >> v1:172.21.15.79:6789/0 conn(0x7f3af815a290 legacy=0x7f3af815d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3af67fc700 1 -- 172.21.15.79:0/1856741320 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af80cd5b0 con 0x7f3af81607a0 2023-06-23T18:36:21.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3af5ffb700 1 -- 172.21.15.79:0/1856741320 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085123087 0 0) 0x7f3af8160030 con 0x7f3af815a290 2023-06-23T18:36:21.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3af67fc700 1 --2- 172.21.15.79:0/1856741320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af81607a0 0x7f3af806a3a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8009840 tx=0x7f3ae8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d593966442a7b4c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3af5ffb700 1 -- 172.21.15.79:0/1856741320 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8011830 con 0x7f3af81607a0 2023-06-23T18:36:21.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3af5ffb700 1 -- 172.21.15.79:0/1856741320 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae8011990 con 0x7f3af81607a0 2023-06-23T18:36:21.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3af5ffb700 1 -- 172.21.15.79:0/1856741320 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8011d20 con 0x7f3af81607a0 2023-06-23T18:36:21.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3afd7cb700 1 -- 172.21.15.79:0/1856741320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af81607a0 msgr2=0x7f3af806a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3afd7cb700 1 --2- 172.21.15.79:0/1856741320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af81607a0 0x7f3af806a3a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8009840 tx=0x7f3ae8009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:21.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3afd7cb700 1 -- 172.21.15.79:0/1856741320 shutdown_connections 2023-06-23T18:36:21.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3afd7cb700 1 --2- 172.21.15.79:0/1856741320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af8160c20 0x7f3af806a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3afd7cb700 1 --2- 172.21.15.79:0/1856741320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af81607a0 0x7f3af806a3a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3afd7cb700 1 -- 172.21.15.79:0/1856741320 >> 172.21.15.79:0/1856741320 conn(0x7f3af80cf790 msgr2=0x7f3af80cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:21.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3afd7cb700 1 -- 172.21.15.79:0/1856741320 shutdown_connections 2023-06-23T18:36:21.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.120+0000 7f3afd7cb700 1 -- 172.21.15.79:0/1856741320 wait complete. 2023-06-23T18:36:21.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.122+0000 7f3afd7cb700 1 Processor -- start 2023-06-23T18:36:21.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.122+0000 7f3afd7cb700 1 -- start start 2023-06-23T18:36:21.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.122+0000 7f3afd7cb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af815a290 0x7f3af80ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.122+0000 7f3afd7cb700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af81607a0 0x7f3af806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.122+0000 7f3afd7cb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3af8160c20 0x7f3af81538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.122+0000 7f3afd7cb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80ad570 con 0x7f3af815a290 2023-06-23T18:36:21.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3afd7cb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af81f2f80 con 0x7f3af8160c20 2023-06-23T18:36:21.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3afd7cb700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80ad780 con 0x7f3af81607a0 2023-06-23T18:36:21.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3af6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af815a290 0x7f3af80ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3af6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af815a290 0x7f3af80ae260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40082/0 (socket says 172.21.15.79:40082) 2023-06-23T18:36:21.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3af6ffd700 1 -- 172.21.15.79:0/138760355 learned_addr learned my addr 172.21.15.79:0/138760355 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:21.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3af67fc700 1 --2- 172.21.15.79:0/138760355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af81607a0 0x7f3af806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3af6ffd700 1 -- 172.21.15.79:0/138760355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af81607a0 msgr2=0x7f3af806b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3af6ffd700 1 --2- 172.21.15.79:0/138760355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af81607a0 0x7f3af806b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3af6ffd700 1 -- 172.21.15.79:0/138760355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3af8160c20 msgr2=0x7f3af81538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3af77fe700 1 --2- 172.21.15.79:0/138760355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3af8160c20 0x7f3af81538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3af6ffd700 1 --2- 172.21.15.79:0/138760355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3af8160c20 0x7f3af81538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3af6ffd700 1 -- 172.21.15.79:0/138760355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae80094a0 con 0x7f3af815a290 2023-06-23T18:36:21.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3af6ffd700 1 --2- 172.21.15.79:0/138760355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af815a290 0x7f3af80ae260 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0011e0 tx=0x7f3aec00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3adf7fe700 1 -- 172.21.15.79:0/138760355 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec004c80 con 0x7f3af815a290 2023-06-23T18:36:21.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3adf7fe700 1 -- 172.21.15.79:0/138760355 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec004e00 con 0x7f3af815a290 2023-06-23T18:36:21.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3afd7cb700 1 -- 172.21.15.79:0/138760355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af80ada60 con 0x7f3af815a290 2023-06-23T18:36:21.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3adf7fe700 1 -- 172.21.15.79:0/138760355 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec007310 con 0x7f3af815a290 2023-06-23T18:36:21.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.123+0000 7f3afd7cb700 1 -- 172.21.15.79:0/138760355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af80ae7a0 con 0x7f3af815a290 2023-06-23T18:36:21.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.124+0000 7f3afd7cb700 1 -- 172.21.15.79:0/138760355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af80672c0 con 0x7f3af815a290 2023-06-23T18:36:21.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.125+0000 7f3adf7fe700 1 -- 172.21.15.79:0/138760355 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3aec007470 con 0x7f3af815a290 2023-06-23T18:36:21.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.125+0000 7f3adf7fe700 1 --2- 172.21.15.79:0/138760355 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3ae406ace0 0x7f3ae406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.125+0000 7f3adf7fe700 1 -- 172.21.15.79:0/138760355 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3aec089220 con 0x7f3af815a290 2023-06-23T18:36:21.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.125+0000 7f3af67fc700 1 --2- 172.21.15.79:0/138760355 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3ae406ace0 0x7f3ae406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.125+0000 7f3af67fc700 1 --2- 172.21.15.79:0/138760355 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3ae406ace0 0x7f3ae406d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3af8154b60 tx=0x7f3ae8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.129+0000 7f3adf7fe700 1 -- 172.21.15.79:0/138760355 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3aec0542d0 con 0x7f3af815a290 2023-06-23T18:36:21.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.314+0000 7f3afd7cb700 1 -- 172.21.15.79:0/138760355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3af8072030 con 0x7f3af815a290 2023-06-23T18:36:21.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.314+0000 7f3adf7fe700 1 -- 172.21.15.79:0/138760355 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3aec0578f0 con 0x7f3af815a290 2023-06-23T18:36:21.320 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:21.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.316+0000 7f3afd7cb700 1 -- 172.21.15.79:0/138760355 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3ae406ace0 msgr2=0x7f3ae406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.316+0000 7f3afd7cb700 1 --2- 172.21.15.79:0/138760355 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3ae406ace0 0x7f3ae406d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3af8154b60 tx=0x7f3ae8008030 comp rx=0 tx=0).stop 2023-06-23T18:36:21.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.316+0000 7f3afd7cb700 1 -- 172.21.15.79:0/138760355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af815a290 msgr2=0x7f3af80ae260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.316+0000 7f3afd7cb700 1 --2- 172.21.15.79:0/138760355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af815a290 0x7f3af80ae260 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0011e0 tx=0x7f3aec00cae0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.316+0000 7f3afd7cb700 1 -- 172.21.15.79:0/138760355 shutdown_connections 2023-06-23T18:36:21.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.316+0000 7f3afd7cb700 1 --2- 172.21.15.79:0/138760355 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3ae406ace0 0x7f3ae406d190 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.316+0000 7f3afd7cb700 1 --2- 172.21.15.79:0/138760355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3af8160c20 0x7f3af81538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.316+0000 7f3afd7cb700 1 --2- 172.21.15.79:0/138760355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3af815a290 0x7f3af80ae260 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.316+0000 7f3afd7cb700 1 --2- 172.21.15.79:0/138760355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af81607a0 0x7f3af806b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.316+0000 7f3afd7cb700 1 -- 172.21.15.79:0/138760355 >> 172.21.15.79:0/138760355 conn(0x7f3af80cf790 msgr2=0x7f3af81550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:21.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.316+0000 7f3afd7cb700 1 -- 172.21.15.79:0/138760355 shutdown_connections 2023-06-23T18:36:21.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.317+0000 7f3afd7cb700 1 -- 172.21.15.79:0/138760355 wait complete. 2023-06-23T18:36:21.333 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:21.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.450+0000 7f4a27a3c700 1 Processor -- start 2023-06-23T18:36:21.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.450+0000 7f4a27a3c700 1 -- start start 2023-06-23T18:36:21.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.451+0000 7f4a27a3c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a2015a2d0 0x7f4a2006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.451+0000 7f4a27a3c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4a200aee00 0x7f4a20071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.451+0000 7f4a27a3c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4a20072140 0x7f4a200719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.451+0000 7f4a27a3c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a200cc170 con 0x7f4a20072140 2023-06-23T18:36:21.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.451+0000 7f4a27a3c700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a2015a6a0 con 0x7f4a200aee00 2023-06-23T18:36:21.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.451+0000 7f4a27a3c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a200cb910 con 0x7f4a2015a2d0 2023-06-23T18:36:21.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.451+0000 7f4a25fd9700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4a20072140 0x7f4a200719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.451+0000 7f4a25fd9700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4a20072140 0x7f4a200719d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40090/0 (socket says 172.21.15.79:40090) 2023-06-23T18:36:21.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.451+0000 7f4a25fd9700 1 -- 172.21.15.79:0/2515193237 learned_addr learned my addr 172.21.15.79:0/2515193237 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:21.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.452+0000 7f4a24fd7700 1 --2- 172.21.15.79:0/2515193237 >> v2:172.21.15.155:3300/0 conn(0x7f4a200aee00 0x7f4a20071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.452+0000 7f4a257d8700 1 --2- 172.21.15.79:0/2515193237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a2015a2d0 0x7f4a2006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.452+0000 7f4a25fd9700 1 -- 172.21.15.79:0/2515193237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a2015a2d0 msgr2=0x7f4a2006eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:21.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.452+0000 7f4a25fd9700 1 --2- 172.21.15.79:0/2515193237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a2015a2d0 0x7f4a2006eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.452+0000 7f4a25fd9700 1 -- 172.21.15.79:0/2515193237 >> v2:172.21.15.155:3300/0 conn(0x7f4a200aee00 msgr2=0x7f4a20071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:21.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.452+0000 7f4a25fd9700 1 --2- 172.21.15.79:0/2515193237 >> v2:172.21.15.155:3300/0 conn(0x7f4a200aee00 0x7f4a20071490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.452+0000 7f4a25fd9700 1 -- 172.21.15.79:0/2515193237 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a200ca200 con 0x7f4a20072140 2023-06-23T18:36:21.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.452+0000 7f4a25fd9700 1 --2- 172.21.15.79:0/2515193237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a20072140 0x7f4a200719d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4a20071180 tx=0x7f4a1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2391b8a1d99aaaf7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.452+0000 7f4a17fff700 1 -- 172.21.15.79:0/2515193237 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1800e070 con 0x7f4a20072140 2023-06-23T18:36:21.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.452+0000 7f4a17fff700 1 -- 172.21.15.79:0/2515193237 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1800b9c0 con 0x7f4a20072140 2023-06-23T18:36:21.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.452+0000 7f4a17fff700 1 -- 172.21.15.79:0/2515193237 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1800bd70 con 0x7f4a20072140 2023-06-23T18:36:21.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.452+0000 7f4a27a3c700 1 -- 172.21.15.79:0/2515193237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a20072140 msgr2=0x7f4a200719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.452+0000 7f4a27a3c700 1 --2- 172.21.15.79:0/2515193237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a20072140 0x7f4a200719d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4a20071180 tx=0x7f4a1800b330 comp rx=0 tx=0).stop 2023-06-23T18:36:21.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.453+0000 7f4a27a3c700 1 -- 172.21.15.79:0/2515193237 shutdown_connections 2023-06-23T18:36:21.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.453+0000 7f4a27a3c700 1 --2- 172.21.15.79:0/2515193237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a2015a2d0 0x7f4a2006eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.453+0000 7f4a27a3c700 1 --2- 172.21.15.79:0/2515193237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a20072140 0x7f4a200719d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.453+0000 7f4a27a3c700 1 --2- 172.21.15.79:0/2515193237 >> v2:172.21.15.155:3300/0 conn(0x7f4a200aee00 0x7f4a20071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.453+0000 7f4a27a3c700 1 -- 172.21.15.79:0/2515193237 >> 172.21.15.79:0/2515193237 conn(0x7f4a2006e200 msgr2=0x7f4a200cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.453+0000 7f4a27a3c700 1 -- 172.21.15.79:0/2515193237 shutdown_connections 2023-06-23T18:36:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.453+0000 7f4a27a3c700 1 -- 172.21.15.79:0/2515193237 wait complete. 2023-06-23T18:36:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.454+0000 7f4a27a3c700 1 Processor -- start 2023-06-23T18:36:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.454+0000 7f4a27a3c700 1 -- start start 2023-06-23T18:36:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a27a3c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a20072140 0x7f4a200c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a27a3c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a200aee00 0x7f4a20156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a27a3c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a2015a2d0 0x7f4a2014b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a27a3c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2015a9e0 con 0x7f4a20072140 2023-06-23T18:36:21.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a27a3c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2015acc0 con 0x7f4a200aee00 2023-06-23T18:36:21.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a27a3c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2015afa0 con 0x7f4a2015a2d0 2023-06-23T18:36:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a257d8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a20072140 0x7f4a200c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a257d8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a20072140 0x7f4a200c5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40094/0 (socket says 172.21.15.79:40094) 2023-06-23T18:36:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a257d8700 1 -- 172.21.15.79:0/1390880199 learned_addr learned my addr 172.21.15.79:0/1390880199 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a24fd7700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a200aee00 0x7f4a20156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a25fd9700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a2015a2d0 0x7f4a2014b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a257d8700 1 -- 172.21.15.79:0/1390880199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a2015a2d0 msgr2=0x7f4a2014b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a257d8700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a2015a2d0 0x7f4a2014b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a257d8700 1 -- 172.21.15.79:0/1390880199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a200aee00 msgr2=0x7f4a20156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a257d8700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a200aee00 0x7f4a20156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a257d8700 1 -- 172.21.15.79:0/1390880199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a18009b30 con 0x7f4a20072140 2023-06-23T18:36:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a24fd7700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a200aee00 0x7f4a20156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a257d8700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a20072140 0x7f4a200c5380 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000c7f0 tx=0x7f4a1000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a15ffb700 1 -- 172.21.15.79:0/1390880199 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10011800 con 0x7f4a20072140 2023-06-23T18:36:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a15ffb700 1 -- 172.21.15.79:0/1390880199 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a10011960 con 0x7f4a20072140 2023-06-23T18:36:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a27a3c700 1 -- 172.21.15.79:0/1390880199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2015b280 con 0x7f4a20072140 2023-06-23T18:36:21.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a15ffb700 1 -- 172.21.15.79:0/1390880199 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10011cf0 con 0x7f4a20072140 2023-06-23T18:36:21.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.455+0000 7f4a27a3c700 1 -- 172.21.15.79:0/1390880199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a200c58c0 con 0x7f4a20072140 2023-06-23T18:36:21.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.456+0000 7f4a27a3c700 1 -- 172.21.15.79:0/1390880199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a200672c0 con 0x7f4a20072140 2023-06-23T18:36:21.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.457+0000 7f4a15ffb700 1 -- 172.21.15.79:0/1390880199 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a10018360 con 0x7f4a20072140 2023-06-23T18:36:21.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.457+0000 7f4a15ffb700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4a0406ace0 0x7f4a0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.457+0000 7f4a15ffb700 1 -- 172.21.15.79:0/1390880199 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4a100893e0 con 0x7f4a20072140 2023-06-23T18:36:21.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.457+0000 7f4a24fd7700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4a0406ace0 0x7f4a0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.457+0000 7f4a24fd7700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4a0406ace0 0x7f4a0406d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4a200c66a0 tx=0x7f4a1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.461+0000 7f4a15ffb700 1 -- 172.21.15.79:0/1390880199 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4a10054490 con 0x7f4a20072140 2023-06-23T18:36:21.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.646+0000 7f4a27a3c700 1 -- 172.21.15.79:0/1390880199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a200ae9d0 con 0x7f4a20072140 2023-06-23T18:36:21.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.646+0000 7f4a15ffb700 1 -- 172.21.15.79:0/1390880199 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a10057ab0 con 0x7f4a20072140 2023-06-23T18:36:21.651 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:21.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.649+0000 7f4a27a3c700 1 -- 172.21.15.79:0/1390880199 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4a0406ace0 msgr2=0x7f4a0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.649+0000 7f4a27a3c700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4a0406ace0 0x7f4a0406d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4a200c66a0 tx=0x7f4a1c008040 comp rx=0 tx=0).stop 2023-06-23T18:36:21.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.649+0000 7f4a27a3c700 1 -- 172.21.15.79:0/1390880199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a20072140 msgr2=0x7f4a200c5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.649+0000 7f4a27a3c700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a20072140 0x7f4a200c5380 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000c7f0 tx=0x7f4a1000cbb0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.649+0000 7f4a27a3c700 1 -- 172.21.15.79:0/1390880199 shutdown_connections 2023-06-23T18:36:21.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.649+0000 7f4a27a3c700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a2015a2d0 0x7f4a2014b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.649+0000 7f4a27a3c700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a20072140 0x7f4a200c5380 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.649+0000 7f4a27a3c700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4a0406ace0 0x7f4a0406d190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.649+0000 7f4a27a3c700 1 --2- 172.21.15.79:0/1390880199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a200aee00 0x7f4a20156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.649+0000 7f4a27a3c700 1 -- 172.21.15.79:0/1390880199 >> 172.21.15.79:0/1390880199 conn(0x7f4a2006e200 msgr2=0x7f4a200cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:21.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.649+0000 7f4a27a3c700 1 -- 172.21.15.79:0/1390880199 shutdown_connections 2023-06-23T18:36:21.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.649+0000 7f4a27a3c700 1 -- 172.21.15.79:0/1390880199 wait complete. 2023-06-23T18:36:21.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:21.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.781+0000 7f447b651700 1 Processor -- start 2023-06-23T18:36:21.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.781+0000 7f447b651700 1 -- start start 2023-06-23T18:36:21.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f447b651700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4474071490 0x7f4474071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f447b651700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f44740b1270 0x7f4474071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f447b651700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44741608f0 con 0x7f44740aee30 2023-06-23T18:36:21.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f447b651700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44740722e0 con 0x7f4474071490 2023-06-23T18:36:21.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f447b651700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4474160e20 con 0x7f44740b1270 2023-06-23T18:36:21.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f44793ed700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4474071490 0x7f4474071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f44793ed700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4474071490 0x7f4474071860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40102/0 (socket says 172.21.15.79:40102) 2023-06-23T18:36:21.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f44793ed700 1 -- 172.21.15.79:0/1828514458 learned_addr learned my addr 172.21.15.79:0/1828514458 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:21.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f4478bec700 1 --2- 172.21.15.79:0/1828514458 >> v2:172.21.15.155:3300/0 conn(0x7f44740b1270 0x7f4474071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f44793ed700 1 -- 172.21.15.79:0/1828514458 >> v2:172.21.15.155:3300/0 conn(0x7f44740b1270 msgr2=0x7f4474071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:21.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f44793ed700 1 --2- 172.21.15.79:0/1828514458 >> v2:172.21.15.155:3300/0 conn(0x7f44740b1270 0x7f4474071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f44793ed700 1 -- 172.21.15.79:0/1828514458 >> v1:172.21.15.155:6789/0 conn(0x7f44740aee30 legacy=0x7f4474069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f44793ed700 1 -- 172.21.15.79:0/1828514458 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44740cd4f0 con 0x7f4474071490 2023-06-23T18:36:21.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f44793ed700 1 --2- 172.21.15.79:0/1828514458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4474071490 0x7f4474071860 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f44740cd230 tx=0x7f4470009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194ca8ab8a78dfad server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f446bfff700 1 -- 172.21.15.79:0/1828514458 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447000b870 con 0x7f4474071490 2023-06-23T18:36:21.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f446bfff700 1 -- 172.21.15.79:0/1828514458 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447000b9d0 con 0x7f4474071490 2023-06-23T18:36:21.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.782+0000 7f446bfff700 1 -- 172.21.15.79:0/1828514458 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447000bd60 con 0x7f4474071490 2023-06-23T18:36:21.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.783+0000 7f447b651700 1 -- 172.21.15.79:0/1828514458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4474071490 msgr2=0x7f4474071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.783+0000 7f447b651700 1 --2- 172.21.15.79:0/1828514458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4474071490 0x7f4474071860 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f44740cd230 tx=0x7f4470009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.783+0000 7f447b651700 1 -- 172.21.15.79:0/1828514458 shutdown_connections 2023-06-23T18:36:21.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.783+0000 7f447b651700 1 --2- 172.21.15.79:0/1828514458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4474071490 0x7f4474071860 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.783+0000 7f447b651700 1 --2- 172.21.15.79:0/1828514458 >> v2:172.21.15.155:3300/0 conn(0x7f44740b1270 0x7f4474071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.783+0000 7f447b651700 1 -- 172.21.15.79:0/1828514458 >> 172.21.15.79:0/1828514458 conn(0x7f447406d660 msgr2=0x7f447406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:21.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.783+0000 7f447b651700 1 -- 172.21.15.79:0/1828514458 shutdown_connections 2023-06-23T18:36:21.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.783+0000 7f447b651700 1 -- 172.21.15.79:0/1828514458 wait complete. 2023-06-23T18:36:21.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.784+0000 7f447b651700 1 Processor -- start 2023-06-23T18:36:21.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.784+0000 7f447b651700 1 -- start start 2023-06-23T18:36:21.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f447b651700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4474071490 0x7f44741578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f447b651700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44740aee30 0x7f44741f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f447b651700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44740b1270 0x7f44740ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f447b651700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44741f3840 con 0x7f44740b1270 2023-06-23T18:36:21.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f447b651700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44741f3600 con 0x7f44740aee30 2023-06-23T18:36:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f447b651700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44741f3040 con 0x7f4474071490 2023-06-23T18:36:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f4479bee700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44740b1270 0x7f44740ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f4479bee700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44740b1270 0x7f44740ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40112/0 (socket says 172.21.15.79:40112) 2023-06-23T18:36:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f4479bee700 1 -- 172.21.15.79:0/1806649486 learned_addr learned my addr 172.21.15.79:0/1806649486 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f4478bec700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44740aee30 0x7f44741f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f44793ed700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4474071490 0x7f44741578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f4479bee700 1 -- 172.21.15.79:0/1806649486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4474071490 msgr2=0x7f44741578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f4479bee700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4474071490 0x7f44741578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f4479bee700 1 -- 172.21.15.79:0/1806649486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44740aee30 msgr2=0x7f44741f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f4479bee700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44740aee30 0x7f44741f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f4479bee700 1 -- 172.21.15.79:0/1806649486 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4470009550 con 0x7f44740b1270 2023-06-23T18:36:21.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f4478bec700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44740aee30 0x7f44741f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:21.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.785+0000 7f44793ed700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4474071490 0x7f44741578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:21.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.786+0000 7f4479bee700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44740b1270 0x7f44740ab8b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f446c00cb50 tx=0x7f446c00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.786+0000 7f4469ffb700 1 -- 172.21.15.79:0/1806649486 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c003c00 con 0x7f44740b1270 2023-06-23T18:36:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.786+0000 7f4469ffb700 1 -- 172.21.15.79:0/1806649486 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c003d60 con 0x7f44740b1270 2023-06-23T18:36:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.786+0000 7f447b651700 1 -- 172.21.15.79:0/1806649486 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44741f3320 con 0x7f44740b1270 2023-06-23T18:36:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.786+0000 7f4469ffb700 1 -- 172.21.15.79:0/1806649486 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c00ab70 con 0x7f44740b1270 2023-06-23T18:36:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.786+0000 7f447b651700 1 -- 172.21.15.79:0/1806649486 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447415d6e0 con 0x7f44740b1270 2023-06-23T18:36:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.787+0000 7f447b651700 1 -- 172.21.15.79:0/1806649486 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44740cdd20 con 0x7f44740b1270 2023-06-23T18:36:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.787+0000 7f4469ffb700 1 -- 172.21.15.79:0/1806649486 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f446c00acd0 con 0x7f44740b1270 2023-06-23T18:36:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.787+0000 7f4469ffb700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f445806ace0 0x7f445806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.788+0000 7f4469ffb700 1 -- 172.21.15.79:0/1806649486 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f446c089520 con 0x7f44740b1270 2023-06-23T18:36:21.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.788+0000 7f44793ed700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f445806ace0 0x7f445806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.788+0000 7f44793ed700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f445806ace0 0x7f445806d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4470014040 tx=0x7f4470008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.792+0000 7f4469ffb700 1 -- 172.21.15.79:0/1806649486 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f446c0545d0 con 0x7f44740b1270 2023-06-23T18:36:21.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.980+0000 7f447b651700 1 -- 172.21.15.79:0/1806649486 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44740724a0 con 0x7f44740b1270 2023-06-23T18:36:21.987 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.981+0000 7f4469ffb700 1 -- 172.21.15.79:0/1806649486 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7f446c057bf0 con 0x7f44740b1270 2023-06-23T18:36:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:21.988 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:21.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.983+0000 7f447b651700 1 -- 172.21.15.79:0/1806649486 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f445806ace0 msgr2=0x7f445806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.983+0000 7f447b651700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f445806ace0 0x7f445806d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4470014040 tx=0x7f4470008030 comp rx=0 tx=0).stop 2023-06-23T18:36:21.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.983+0000 7f447b651700 1 -- 172.21.15.79:0/1806649486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44740b1270 msgr2=0x7f44740ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.983+0000 7f447b651700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44740b1270 0x7f44740ab8b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f446c00cb50 tx=0x7f446c00cf10 comp rx=0 tx=0).stop 2023-06-23T18:36:21.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.983+0000 7f447b651700 1 -- 172.21.15.79:0/1806649486 shutdown_connections 2023-06-23T18:36:21.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.983+0000 7f447b651700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44740b1270 0x7f44740ab8b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.983+0000 7f447b651700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4474071490 0x7f44741578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.983+0000 7f447b651700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f445806ace0 0x7f445806d190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.983+0000 7f447b651700 1 --2- 172.21.15.79:0/1806649486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44740aee30 0x7f44741f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.983+0000 7f447b651700 1 -- 172.21.15.79:0/1806649486 >> 172.21.15.79:0/1806649486 conn(0x7f447406d660 msgr2=0x7f44740caac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:21.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.983+0000 7f447b651700 1 -- 172.21.15.79:0/1806649486 shutdown_connections 2023-06-23T18:36:21.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:21.983+0000 7f447b651700 1 -- 172.21.15.79:0/1806649486 wait complete. 2023-06-23T18:36:22.000 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:22.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:22.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.118+0000 7f26d3a36700 1 Processor -- start 2023-06-23T18:36:22.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.119+0000 7f26d3a36700 1 -- start start 2023-06-23T18:36:22.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.119+0000 7f26d3a36700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f26cc1569c0 0x7f26cc1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.119+0000 7f26d3a36700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f26cc06a5a0 0x7f26cc0b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.119+0000 7f26d3a36700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26cc0cdc90 con 0x7f26cc0cd440 2023-06-23T18:36:22.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.119+0000 7f26d3a36700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26cc0cf570 con 0x7f26cc1569c0 2023-06-23T18:36:22.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.119+0000 7f26d3a36700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26cc15c8b0 con 0x7f26cc06a5a0 2023-06-23T18:36:22.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.120+0000 7f26d17d2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f26cc1569c0 0x7f26cc1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.120+0000 7f26d0fd1700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f26cc0cd440 0x7f26cc069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43158/0 (socket says 172.21.15.79:43158) 2023-06-23T18:36:22.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.120+0000 7f26d0fd1700 1 -- 172.21.15.79:0/850156080 learned_addr learned my addr 172.21.15.79:0/850156080 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:22.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.120+0000 7f26d1fd3700 1 --2- 172.21.15.79:0/850156080 >> v2:172.21.15.155:3300/0 conn(0x7f26cc06a5a0 0x7f26cc0b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.120+0000 7f26d17d2700 1 -- 172.21.15.79:0/850156080 >> v2:172.21.15.155:3300/0 conn(0x7f26cc06a5a0 msgr2=0x7f26cc0b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:22.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.120+0000 7f26d17d2700 1 --2- 172.21.15.79:0/850156080 >> v2:172.21.15.155:3300/0 conn(0x7f26cc06a5a0 0x7f26cc0b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.120+0000 7f26d17d2700 1 -- 172.21.15.79:0/850156080 >> v1:172.21.15.79:6789/0 conn(0x7f26cc0cd440 legacy=0x7f26cc069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.120+0000 7f26c3fff700 1 -- 172.21.15.79:0/850156080 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795928263 0 0) 0x7f26c8001110 con 0x7f26cc0cd440 2023-06-23T18:36:22.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.120+0000 7f26d17d2700 1 -- 172.21.15.79:0/850156080 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc15d0f0 con 0x7f26cc1569c0 2023-06-23T18:36:22.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.120+0000 7f26d17d2700 1 --2- 172.21.15.79:0/850156080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1569c0 0x7f26cc1608b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f26cc06e640 tx=0x7f26bc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5df9491fbaa6ff3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:22.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.120+0000 7f26c3fff700 1 -- 172.21.15.79:0/850156080 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc011830 con 0x7f26cc1569c0 2023-06-23T18:36:22.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.120+0000 7f26c3fff700 1 -- 172.21.15.79:0/850156080 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc011990 con 0x7f26cc1569c0 2023-06-23T18:36:22.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.120+0000 7f26c3fff700 1 -- 172.21.15.79:0/850156080 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc011d20 con 0x7f26cc1569c0 2023-06-23T18:36:22.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.121+0000 7f26d3a36700 1 -- 172.21.15.79:0/850156080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1569c0 msgr2=0x7f26cc1608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.121+0000 7f26d3a36700 1 --2- 172.21.15.79:0/850156080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1569c0 0x7f26cc1608b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f26cc06e640 tx=0x7f26bc009b20 comp rx=0 tx=0).stop 2023-06-23T18:36:22.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.121+0000 7f26d3a36700 1 -- 172.21.15.79:0/850156080 shutdown_connections 2023-06-23T18:36:22.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.121+0000 7f26d3a36700 1 --2- 172.21.15.79:0/850156080 >> v2:172.21.15.155:3300/0 conn(0x7f26cc06a5a0 0x7f26cc0b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.121+0000 7f26d3a36700 1 --2- 172.21.15.79:0/850156080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1569c0 0x7f26cc1608b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.121+0000 7f26d3a36700 1 -- 172.21.15.79:0/850156080 >> 172.21.15.79:0/850156080 conn(0x7f26cc06afa0 msgr2=0x7f26cc06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:22.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.121+0000 7f26d3a36700 1 -- 172.21.15.79:0/850156080 shutdown_connections 2023-06-23T18:36:22.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.121+0000 7f26d3a36700 1 -- 172.21.15.79:0/850156080 wait complete. 2023-06-23T18:36:22.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.122+0000 7f26d3a36700 1 Processor -- start 2023-06-23T18:36:22.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.122+0000 7f26d3a36700 1 -- start start 2023-06-23T18:36:22.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d3a36700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26cc06a5a0 0x7f26cc14bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d3a36700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26cc0cd440 0x7f26cc0b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d3a36700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1569c0 0x7f26cc0ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d3a36700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc0b3110 con 0x7f26cc1569c0 2023-06-23T18:36:22.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d3a36700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc14c2a0 con 0x7f26cc0cd440 2023-06-23T18:36:22.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d3a36700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc0b5760 con 0x7f26cc06a5a0 2023-06-23T18:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d1fd3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1569c0 0x7f26cc0ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d1fd3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1569c0 0x7f26cc0ac900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40126/0 (socket says 172.21.15.79:40126) 2023-06-23T18:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d1fd3700 1 -- 172.21.15.79:0/3028782199 learned_addr learned my addr 172.21.15.79:0/3028782199 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d0fd1700 1 --2- 172.21.15.79:0/3028782199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26cc0cd440 0x7f26cc0b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d17d2700 1 --2- 172.21.15.79:0/3028782199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26cc06a5a0 0x7f26cc14bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d1fd3700 1 -- 172.21.15.79:0/3028782199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26cc06a5a0 msgr2=0x7f26cc14bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d1fd3700 1 --2- 172.21.15.79:0/3028782199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26cc06a5a0 0x7f26cc14bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d1fd3700 1 -- 172.21.15.79:0/3028782199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26cc0cd440 msgr2=0x7f26cc0b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d1fd3700 1 --2- 172.21.15.79:0/3028782199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26cc0cd440 0x7f26cc0b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d1fd3700 1 -- 172.21.15.79:0/3028782199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc0094a0 con 0x7f26cc1569c0 2023-06-23T18:36:22.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d1fd3700 1 --2- 172.21.15.79:0/3028782199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1569c0 0x7f26cc0ac900 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f26c4007e30 tx=0x7f26c400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:22.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26c1ffb700 1 -- 172.21.15.79:0/3028782199 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c400eea0 con 0x7f26cc1569c0 2023-06-23T18:36:22.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26c1ffb700 1 -- 172.21.15.79:0/3028782199 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c4014420 con 0x7f26cc1569c0 2023-06-23T18:36:22.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26d3a36700 1 -- 172.21.15.79:0/3028782199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26cc0c9640 con 0x7f26cc1569c0 2023-06-23T18:36:22.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.123+0000 7f26c1ffb700 1 -- 172.21.15.79:0/3028782199 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c40147d0 con 0x7f26cc1569c0 2023-06-23T18:36:22.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.124+0000 7f26d3a36700 1 -- 172.21.15.79:0/3028782199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26cc0b2a50 con 0x7f26cc1569c0 2023-06-23T18:36:22.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.124+0000 7f26d3a36700 1 -- 172.21.15.79:0/3028782199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26cc0b3250 con 0x7f26cc1569c0 2023-06-23T18:36:22.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.125+0000 7f26c1ffb700 1 -- 172.21.15.79:0/3028782199 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f26c4014930 con 0x7f26cc1569c0 2023-06-23T18:36:22.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.125+0000 7f26c1ffb700 1 --2- 172.21.15.79:0/3028782199 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f26b006ace0 0x7f26b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.125+0000 7f26c1ffb700 1 -- 172.21.15.79:0/3028782199 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f26c4089c60 con 0x7f26cc1569c0 2023-06-23T18:36:22.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.125+0000 7f26d17d2700 1 --2- 172.21.15.79:0/3028782199 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f26b006ace0 0x7f26b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.126+0000 7f26d17d2700 1 --2- 172.21.15.79:0/3028782199 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f26b006ace0 0x7f26b006d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0090b0 tx=0x7f26bc01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:22.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.129+0000 7f26c1ffb700 1 -- 172.21.15.79:0/3028782199 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f26cc0b3250 con 0x7f26cc1569c0 2023-06-23T18:36:22.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.314+0000 7f26d3a36700 1 -- 172.21.15.79:0/3028782199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f26cc0b3250 con 0x7f26cc1569c0 2023-06-23T18:36:22.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.314+0000 7f26c1ffb700 1 -- 172.21.15.79:0/3028782199 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f26cc0b3250 con 0x7f26cc1569c0 2023-06-23T18:36:22.319 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:22.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.316+0000 7f26d3a36700 1 -- 172.21.15.79:0/3028782199 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f26b006ace0 msgr2=0x7f26b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.316+0000 7f26d3a36700 1 --2- 172.21.15.79:0/3028782199 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f26b006ace0 0x7f26b006d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0090b0 tx=0x7f26bc01c040 comp rx=0 tx=0).stop 2023-06-23T18:36:22.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.316+0000 7f26d3a36700 1 -- 172.21.15.79:0/3028782199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1569c0 msgr2=0x7f26cc0ac900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.316+0000 7f26d3a36700 1 --2- 172.21.15.79:0/3028782199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1569c0 0x7f26cc0ac900 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f26c4007e30 tx=0x7f26c400e3c0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.316+0000 7f26d3a36700 1 -- 172.21.15.79:0/3028782199 shutdown_connections 2023-06-23T18:36:22.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.316+0000 7f26d3a36700 1 --2- 172.21.15.79:0/3028782199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1569c0 0x7f26cc0ac900 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.316+0000 7f26d3a36700 1 --2- 172.21.15.79:0/3028782199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26cc06a5a0 0x7f26cc14bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.316+0000 7f26d3a36700 1 --2- 172.21.15.79:0/3028782199 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f26b006ace0 0x7f26b006d190 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.316+0000 7f26d3a36700 1 --2- 172.21.15.79:0/3028782199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26cc0cd440 0x7f26cc0b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.316+0000 7f26d3a36700 1 -- 172.21.15.79:0/3028782199 >> 172.21.15.79:0/3028782199 conn(0x7f26cc06afa0 msgr2=0x7f26cc15f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:22.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.316+0000 7f26d3a36700 1 -- 172.21.15.79:0/3028782199 shutdown_connections 2023-06-23T18:36:22.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.316+0000 7f26d3a36700 1 -- 172.21.15.79:0/3028782199 wait complete. 2023-06-23T18:36:22.333 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:22.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.450+0000 7f5ea95a3700 1 Processor -- start 2023-06-23T18:36:22.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.450+0000 7f5ea95a3700 1 -- start start 2023-06-23T18:36:22.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.450+0000 7f5ea95a3700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415c750 0x7f5ea40ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.451+0000 7f5ea95a3700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea40cf690 con 0x7f5ea40ac450 2023-06-23T18:36:22.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.451+0000 7f5ea95a3700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea40ac8c0 con 0x7f5ea40ad960 2023-06-23T18:36:22.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.451+0000 7f5ea95a3700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea40acba0 con 0x7f5ea415c750 2023-06-23T18:36:22.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.451+0000 7f5ea27fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5ea40ac450 0x7f5ea40c9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43172/0 (socket says 172.21.15.79:43172) 2023-06-23T18:36:22.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.451+0000 7f5ea27fc700 1 -- 172.21.15.79:0/2376698220 learned_addr learned my addr 172.21.15.79:0/2376698220 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:22.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.451+0000 7f5ea2ffd700 1 --2- 172.21.15.79:0/2376698220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415c750 0x7f5ea40ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.451+0000 7f5ea1ffb700 1 -- 172.21.15.79:0/2376698220 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743015014 0 0) 0x7f5ea40cf690 con 0x7f5ea40ac450 2023-06-23T18:36:22.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.451+0000 7f5ea1ffb700 1 -- 172.21.15.79:0/2376698220 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e8c0035b0 con 0x7f5ea40ac450 2023-06-23T18:36:22.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.451+0000 7f5ea2ffd700 1 -- 172.21.15.79:0/2376698220 >> v1:172.21.15.155:6789/0 conn(0x7f5ea40ad960 legacy=0x7f5ea4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.451+0000 7f5ea2ffd700 1 -- 172.21.15.79:0/2376698220 >> v1:172.21.15.79:6789/0 conn(0x7f5ea40ac450 legacy=0x7f5ea40c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.451+0000 7f5ea2ffd700 1 -- 172.21.15.79:0/2376698220 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea40ace80 con 0x7f5ea415c750 2023-06-23T18:36:22.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.451+0000 7f5ea2ffd700 1 --2- 172.21.15.79:0/2376698220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415c750 0x7f5ea40ce470 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009850 tx=0x7f5e94009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91581f5a1b867ea5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:22.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.452+0000 7f5ea1ffb700 1 -- 172.21.15.79:0/2376698220 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94005e30 con 0x7f5ea415c750 2023-06-23T18:36:22.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.452+0000 7f5ea1ffb700 1 -- 172.21.15.79:0/2376698220 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e94006030 con 0x7f5ea415c750 2023-06-23T18:36:22.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.452+0000 7f5ea1ffb700 1 -- 172.21.15.79:0/2376698220 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e940063e0 con 0x7f5ea415c750 2023-06-23T18:36:22.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.452+0000 7f5ea95a3700 1 -- 172.21.15.79:0/2376698220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415c750 msgr2=0x7f5ea40ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.452+0000 7f5ea95a3700 1 --2- 172.21.15.79:0/2376698220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415c750 0x7f5ea40ce470 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009850 tx=0x7f5e94009b60 comp rx=0 tx=0).stop 2023-06-23T18:36:22.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.452+0000 7f5ea95a3700 1 -- 172.21.15.79:0/2376698220 shutdown_connections 2023-06-23T18:36:22.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.452+0000 7f5ea95a3700 1 --2- 172.21.15.79:0/2376698220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415c750 0x7f5ea40ce470 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.452+0000 7f5ea95a3700 1 -- 172.21.15.79:0/2376698220 >> 172.21.15.79:0/2376698220 conn(0x7f5ea406b8c0 msgr2=0x7f5ea40cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:22.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.452+0000 7f5ea95a3700 1 -- 172.21.15.79:0/2376698220 shutdown_connections 2023-06-23T18:36:22.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.452+0000 7f5ea95a3700 1 -- 172.21.15.79:0/2376698220 wait complete. 2023-06-23T18:36:22.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.453+0000 7f5ea95a3700 1 Processor -- start 2023-06-23T18:36:22.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.453+0000 7f5ea95a3700 1 -- start start 2023-06-23T18:36:22.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.454+0000 7f5ea95a3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ea40ac450 0x7f5ea415aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.454+0000 7f5ea95a3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ea40ad960 0x7f5ea415a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.454+0000 7f5ea95a3700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415c750 0x7f5ea40addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.454+0000 7f5ea95a3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea415db70 con 0x7f5ea40ad960 2023-06-23T18:36:22.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.454+0000 7f5ea95a3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea415e410 con 0x7f5ea40ac450 2023-06-23T18:36:22.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.454+0000 7f5ea95a3700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea40c8e50 con 0x7f5ea415c750 2023-06-23T18:36:22.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.454+0000 7f5ea27fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ea40ad960 0x7f5ea415a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.454+0000 7f5ea27fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ea40ad960 0x7f5ea415a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40138/0 (socket says 172.21.15.79:40138) 2023-06-23T18:36:22.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.454+0000 7f5ea27fc700 1 -- 172.21.15.79:0/3805880738 learned_addr learned my addr 172.21.15.79:0/3805880738 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:22.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.454+0000 7f5ea2ffd700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ea40ac450 0x7f5ea415aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.454+0000 7f5ea37fe700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415c750 0x7f5ea40addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.455+0000 7f5ea27fc700 1 -- 172.21.15.79:0/3805880738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415c750 msgr2=0x7f5ea40addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.455+0000 7f5ea27fc700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415c750 0x7f5ea40addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.455+0000 7f5ea27fc700 1 -- 172.21.15.79:0/3805880738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ea40ac450 msgr2=0x7f5ea415aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.455+0000 7f5ea27fc700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ea40ac450 0x7f5ea415aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.455+0000 7f5ea27fc700 1 -- 172.21.15.79:0/3805880738 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e94009430 con 0x7f5ea40ad960 2023-06-23T18:36:22.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.455+0000 7f5ea2ffd700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ea40ac450 0x7f5ea415aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:22.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.455+0000 7f5ea27fc700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ea40ad960 0x7f5ea415a960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800c700 tx=0x7f5e9800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:22.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.455+0000 7f5e8b7fe700 1 -- 172.21.15.79:0/3805880738 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98005d70 con 0x7f5ea40ad960 2023-06-23T18:36:22.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.455+0000 7f5e8b7fe700 1 -- 172.21.15.79:0/3805880738 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e980070d0 con 0x7f5ea40ad960 2023-06-23T18:36:22.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.455+0000 7f5e8b7fe700 1 -- 172.21.15.79:0/3805880738 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98007460 con 0x7f5ea40ad960 2023-06-23T18:36:22.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.455+0000 7f5ea95a3700 1 -- 172.21.15.79:0/3805880738 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea40c9130 con 0x7f5ea40ad960 2023-06-23T18:36:22.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.455+0000 7f5ea95a3700 1 -- 172.21.15.79:0/3805880738 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea415e0a0 con 0x7f5ea40ad960 2023-06-23T18:36:22.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.456+0000 7f5ea95a3700 1 -- 172.21.15.79:0/3805880738 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea40c9410 con 0x7f5ea40ad960 2023-06-23T18:36:22.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.456+0000 7f5e8b7fe700 1 -- 172.21.15.79:0/3805880738 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5e980075c0 con 0x7f5ea40ad960 2023-06-23T18:36:22.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.456+0000 7f5e8b7fe700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5e9006ace0 0x7f5e9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.457+0000 7f5e8b7fe700 1 -- 172.21.15.79:0/3805880738 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5e98088ad0 con 0x7f5ea40ad960 2023-06-23T18:36:22.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.457+0000 7f5ea2ffd700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5e9006ace0 0x7f5e9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.457+0000 7f5ea2ffd700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5e9006ace0 0x7f5e9006d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e94013040 tx=0x7f5e94008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:22.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.461+0000 7f5e8b7fe700 1 -- 172.21.15.79:0/3805880738 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5e98053b90 con 0x7f5ea40ad960 2023-06-23T18:36:22.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.646+0000 7f5ea95a3700 1 -- 172.21.15.79:0/3805880738 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5ea404e1d0 con 0x7f5ea40ad960 2023-06-23T18:36:22.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.647+0000 7f5e8b7fe700 1 -- 172.21.15.79:0/3805880738 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e980571b0 con 0x7f5ea40ad960 2023-06-23T18:36:22.652 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:22.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.649+0000 7f5ea95a3700 1 -- 172.21.15.79:0/3805880738 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5e9006ace0 msgr2=0x7f5e9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.649+0000 7f5ea95a3700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5e9006ace0 0x7f5e9006d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e94013040 tx=0x7f5e94008070 comp rx=0 tx=0).stop 2023-06-23T18:36:22.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.649+0000 7f5ea95a3700 1 -- 172.21.15.79:0/3805880738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ea40ad960 msgr2=0x7f5ea415a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.649+0000 7f5ea95a3700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ea40ad960 0x7f5ea415a960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800c700 tx=0x7f5e9800cac0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.649+0000 7f5ea95a3700 1 -- 172.21.15.79:0/3805880738 shutdown_connections 2023-06-23T18:36:22.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.649+0000 7f5ea95a3700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415c750 0x7f5ea40addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.649+0000 7f5ea95a3700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5e9006ace0 0x7f5e9006d190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.649+0000 7f5ea95a3700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ea40ac450 0x7f5ea415aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.649+0000 7f5ea95a3700 1 --2- 172.21.15.79:0/3805880738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ea40ad960 0x7f5ea415a960 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.649+0000 7f5ea95a3700 1 -- 172.21.15.79:0/3805880738 >> 172.21.15.79:0/3805880738 conn(0x7f5ea406b8c0 msgr2=0x7f5ea4069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:22.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.649+0000 7f5ea95a3700 1 -- 172.21.15.79:0/3805880738 shutdown_connections 2023-06-23T18:36:22.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.649+0000 7f5ea95a3700 1 -- 172.21.15.79:0/3805880738 wait complete. 2023-06-23T18:36:22.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:22.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.783+0000 7fc041bcf700 1 Processor -- start 2023-06-23T18:36:22.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.783+0000 7fc041bcf700 1 -- start start 2023-06-23T18:36:22.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.784+0000 7fc041bcf700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc03c06a5a0 0x7fc03c1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.784+0000 7fc041bcf700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc03c0cd440 0x7fc03c0b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.784+0000 7fc041bcf700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc03c0cdc90 con 0x7fc03c1569c0 2023-06-23T18:36:22.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.784+0000 7fc041bcf700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03c0cf570 con 0x7fc03c06a5a0 2023-06-23T18:36:22.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.784+0000 7fc041bcf700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03c15c8b0 con 0x7fc03c0cd440 2023-06-23T18:36:22.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.784+0000 7fc03affd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc03c06a5a0 0x7fc03c1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.784+0000 7fc03b7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fc03c1569c0 0x7fc03c069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43186/0 (socket says 172.21.15.79:43186) 2023-06-23T18:36:22.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.784+0000 7fc03b7fe700 1 -- 172.21.15.79:0/640791411 learned_addr learned my addr 172.21.15.79:0/640791411 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:22.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.784+0000 7fc03bfff700 1 --2- 172.21.15.79:0/640791411 >> v2:172.21.15.155:3300/0 conn(0x7fc03c0cd440 0x7fc03c0b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.784+0000 7fc03affd700 1 -- 172.21.15.79:0/640791411 >> v2:172.21.15.155:3300/0 conn(0x7fc03c0cd440 msgr2=0x7fc03c0b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:22.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.784+0000 7fc03affd700 1 --2- 172.21.15.79:0/640791411 >> v2:172.21.15.155:3300/0 conn(0x7fc03c0cd440 0x7fc03c0b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.784+0000 7fc03affd700 1 -- 172.21.15.79:0/640791411 >> v1:172.21.15.79:6789/0 conn(0x7fc03c1569c0 legacy=0x7fc03c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.785+0000 7fc03affd700 1 -- 172.21.15.79:0/640791411 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c15d0f0 con 0x7fc03c06a5a0 2023-06-23T18:36:22.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.784+0000 7fc03a7fc700 1 -- 172.21.15.79:0/640791411 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1418297701 0 0) 0x7fc030001110 con 0x7fc03c1569c0 2023-06-23T18:36:22.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.785+0000 7fc03bfff700 1 --2- 172.21.15.79:0/640791411 >> v2:172.21.15.155:3300/0 conn(0x7fc03c0cd440 0x7fc03c0b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:22.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.785+0000 7fc03affd700 1 --2- 172.21.15.79:0/640791411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc03c06a5a0 0x7fc03c1608b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc03c06e8f0 tx=0x7fc02c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c83604c584250b2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:22.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.785+0000 7fc03a7fc700 1 -- 172.21.15.79:0/640791411 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c011830 con 0x7fc03c06a5a0 2023-06-23T18:36:22.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.785+0000 7fc03a7fc700 1 -- 172.21.15.79:0/640791411 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c011990 con 0x7fc03c06a5a0 2023-06-23T18:36:22.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.785+0000 7fc03a7fc700 1 -- 172.21.15.79:0/640791411 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c011d20 con 0x7fc03c06a5a0 2023-06-23T18:36:22.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.785+0000 7fc041bcf700 1 -- 172.21.15.79:0/640791411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc03c06a5a0 msgr2=0x7fc03c1608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.785+0000 7fc041bcf700 1 --2- 172.21.15.79:0/640791411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc03c06a5a0 0x7fc03c1608b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc03c06e8f0 tx=0x7fc02c009b20 comp rx=0 tx=0).stop 2023-06-23T18:36:22.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.785+0000 7fc041bcf700 1 -- 172.21.15.79:0/640791411 shutdown_connections 2023-06-23T18:36:22.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.785+0000 7fc041bcf700 1 --2- 172.21.15.79:0/640791411 >> v2:172.21.15.155:3300/0 conn(0x7fc03c0cd440 0x7fc03c0b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.785+0000 7fc041bcf700 1 --2- 172.21.15.79:0/640791411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc03c06a5a0 0x7fc03c1608b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.785+0000 7fc041bcf700 1 -- 172.21.15.79:0/640791411 >> 172.21.15.79:0/640791411 conn(0x7fc03c06afa0 msgr2=0x7fc03c06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:22.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.785+0000 7fc041bcf700 1 -- 172.21.15.79:0/640791411 shutdown_connections 2023-06-23T18:36:22.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.785+0000 7fc041bcf700 1 -- 172.21.15.79:0/640791411 wait complete. 2023-06-23T18:36:22.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.787+0000 7fc041bcf700 1 Processor -- start 2023-06-23T18:36:22.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.787+0000 7fc041bcf700 1 -- start start 2023-06-23T18:36:22.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.787+0000 7fc041bcf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc03c06a5a0 0x7fc03c0b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.787+0000 7fc041bcf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc03c0cd440 0x7fc03c1f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc041bcf700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc03c1569c0 0x7fc03c1f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc041bcf700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0c59f0 con 0x7fc03c06a5a0 2023-06-23T18:36:22.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc041bcf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0ae9b0 con 0x7fc03c0cd440 2023-06-23T18:36:22.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc041bcf700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0c4e00 con 0x7fc03c1569c0 2023-06-23T18:36:22.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc03b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc03c06a5a0 0x7fc03c0b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc03b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc03c06a5a0 0x7fc03c0b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40150/0 (socket says 172.21.15.79:40150) 2023-06-23T18:36:22.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc03b7fe700 1 -- 172.21.15.79:0/2650821879 learned_addr learned my addr 172.21.15.79:0/2650821879 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:22.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc03affd700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc03c0cd440 0x7fc03c1f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc03bfff700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc03c1569c0 0x7fc03c1f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc03b7fe700 1 -- 172.21.15.79:0/2650821879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc03c1569c0 msgr2=0x7fc03c1f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc03b7fe700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc03c1569c0 0x7fc03c1f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc03b7fe700 1 -- 172.21.15.79:0/2650821879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc03c0cd440 msgr2=0x7fc03c1f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc03b7fe700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc03c0cd440 0x7fc03c1f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc03b7fe700 1 -- 172.21.15.79:0/2650821879 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c0094a0 con 0x7fc03c06a5a0 2023-06-23T18:36:22.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc03affd700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc03c0cd440 0x7fc03c1f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:22.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc03b7fe700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc03c06a5a0 0x7fc03c0b26a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc03000c550 tx=0x7fc03000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:22.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc023fff700 1 -- 172.21.15.79:0/2650821879 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030004c80 con 0x7fc03c06a5a0 2023-06-23T18:36:22.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc023fff700 1 -- 172.21.15.79:0/2650821879 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc030004e00 con 0x7fc03c06a5a0 2023-06-23T18:36:22.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc041bcf700 1 -- 172.21.15.79:0/2650821879 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03c0c50c0 con 0x7fc03c06a5a0 2023-06-23T18:36:22.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc023fff700 1 -- 172.21.15.79:0/2650821879 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030007310 con 0x7fc03c06a5a0 2023-06-23T18:36:22.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.788+0000 7fc041bcf700 1 -- 172.21.15.79:0/2650821879 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c0c5680 con 0x7fc03c06a5a0 2023-06-23T18:36:22.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.789+0000 7fc041bcf700 1 -- 172.21.15.79:0/2650821879 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03c0672c0 con 0x7fc03c06a5a0 2023-06-23T18:36:22.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.790+0000 7fc023fff700 1 -- 172.21.15.79:0/2650821879 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc030007470 con 0x7fc03c06a5a0 2023-06-23T18:36:22.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.790+0000 7fc023fff700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc02806ace0 0x7fc02806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:22.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.790+0000 7fc03affd700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc02806ace0 0x7fc02806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:22.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.790+0000 7fc023fff700 1 -- 172.21.15.79:0/2650821879 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc030089000 con 0x7fc03c06a5a0 2023-06-23T18:36:22.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.790+0000 7fc03affd700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc02806ace0 0x7fc02806d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc03c06a990 tx=0x7fc02c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:22.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.794+0000 7fc023fff700 1 -- 172.21.15.79:0/2650821879 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc0300540b0 con 0x7fc03c06a5a0 2023-06-23T18:36:22.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.981+0000 7fc041bcf700 1 -- 172.21.15.79:0/2650821879 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc03c0cdc90 con 0x7fc03c06a5a0 2023-06-23T18:36:22.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.981+0000 7fc023fff700 1 -- 172.21.15.79:0/2650821879 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc03c0cdc90 con 0x7fc03c06a5a0 2023-06-23T18:36:22.986 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:22.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.983+0000 7fc041bcf700 1 -- 172.21.15.79:0/2650821879 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc02806ace0 msgr2=0x7fc02806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.983+0000 7fc041bcf700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc02806ace0 0x7fc02806d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc03c06a990 tx=0x7fc02c008030 comp rx=0 tx=0).stop 2023-06-23T18:36:22.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.983+0000 7fc041bcf700 1 -- 172.21.15.79:0/2650821879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc03c06a5a0 msgr2=0x7fc03c0b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:22.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.983+0000 7fc041bcf700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc03c06a5a0 0x7fc03c0b26a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc03000c550 tx=0x7fc03000c910 comp rx=0 tx=0).stop 2023-06-23T18:36:22.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.983+0000 7fc041bcf700 1 -- 172.21.15.79:0/2650821879 shutdown_connections 2023-06-23T18:36:22.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.983+0000 7fc041bcf700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc03c1569c0 0x7fc03c1f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.983+0000 7fc041bcf700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc03c06a5a0 0x7fc03c0b26a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.983+0000 7fc041bcf700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc02806ace0 0x7fc02806d190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.983+0000 7fc041bcf700 1 --2- 172.21.15.79:0/2650821879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc03c0cd440 0x7fc03c1f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:22.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.983+0000 7fc041bcf700 1 -- 172.21.15.79:0/2650821879 >> 172.21.15.79:0/2650821879 conn(0x7fc03c06afa0 msgr2=0x7fc03c15fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:22.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.983+0000 7fc041bcf700 1 -- 172.21.15.79:0/2650821879 shutdown_connections 2023-06-23T18:36:22.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:22.983+0000 7fc041bcf700 1 -- 172.21.15.79:0/2650821879 wait complete. 2023-06-23T18:36:23.000 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:23.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.118+0000 7f12d8eb6700 1 Processor -- start 2023-06-23T18:36:23.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.118+0000 7f12d8eb6700 1 -- start start 2023-06-23T18:36:23.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.119+0000 7f12d8eb6700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f12d415be70 0x7f12d4159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.119+0000 7f12d8eb6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f12d415c420 0x7f12d40cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.119+0000 7f12d8eb6700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d415d3c0 con 0x7f12d4158db0 2023-06-23T18:36:23.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.119+0000 7f12d8eb6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d415c240 con 0x7f12d415c420 2023-06-23T18:36:23.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.119+0000 7f12d8eb6700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d4159250 con 0x7f12d415be70 2023-06-23T18:36:23.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.119+0000 7f12d2d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f12d415c420 0x7f12d40cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.119+0000 7f12d1d9b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f12d4158db0 0x7f12d4065b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43200/0 (socket says 172.21.15.79:43200) 2023-06-23T18:36:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.119+0000 7f12d1d9b700 1 -- 172.21.15.79:0/2149348172 learned_addr learned my addr 172.21.15.79:0/2149348172 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d2d9d700 1 -- 172.21.15.79:0/2149348172 >> v2:172.21.15.155:3300/0 conn(0x7f12d415be70 msgr2=0x7f12d4159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d259c700 1 --2- 172.21.15.79:0/2149348172 >> v2:172.21.15.155:3300/0 conn(0x7f12d415be70 0x7f12d4159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d2d9d700 1 --2- 172.21.15.79:0/2149348172 >> v2:172.21.15.155:3300/0 conn(0x7f12d415be70 0x7f12d4159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d2d9d700 1 -- 172.21.15.79:0/2149348172 >> v1:172.21.15.79:6789/0 conn(0x7f12d4158db0 legacy=0x7f12d4065b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d2d9d700 1 -- 172.21.15.79:0/2149348172 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d406a0b0 con 0x7f12d415c420 2023-06-23T18:36:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d159a700 1 -- 172.21.15.79:0/2149348172 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4184325249 0 0) 0x7f12d415d3c0 con 0x7f12d4158db0 2023-06-23T18:36:23.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d259c700 1 --2- 172.21.15.79:0/2149348172 >> v2:172.21.15.155:3300/0 conn(0x7f12d415be70 0x7f12d4159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:23.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d2d9d700 1 --2- 172.21.15.79:0/2149348172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12d415c420 0x7f12d40cd040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f12d406e310 tx=0x7f12cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6330d57c295dd7d5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:23.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d159a700 1 -- 172.21.15.79:0/2149348172 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc00e070 con 0x7f12d415c420 2023-06-23T18:36:23.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d159a700 1 -- 172.21.15.79:0/2149348172 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc00bbd0 con 0x7f12d415c420 2023-06-23T18:36:23.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d159a700 1 -- 172.21.15.79:0/2149348172 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc012430 con 0x7f12d415c420 2023-06-23T18:36:23.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d8eb6700 1 -- 172.21.15.79:0/2149348172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12d415c420 msgr2=0x7f12d40cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d8eb6700 1 --2- 172.21.15.79:0/2149348172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12d415c420 0x7f12d40cd040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f12d406e310 tx=0x7f12cc00b330 comp rx=0 tx=0).stop 2023-06-23T18:36:23.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d8eb6700 1 -- 172.21.15.79:0/2149348172 shutdown_connections 2023-06-23T18:36:23.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d8eb6700 1 --2- 172.21.15.79:0/2149348172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12d415c420 0x7f12d40cd040 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d8eb6700 1 --2- 172.21.15.79:0/2149348172 >> v2:172.21.15.155:3300/0 conn(0x7f12d415be70 0x7f12d4159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.120+0000 7f12d8eb6700 1 -- 172.21.15.79:0/2149348172 >> 172.21.15.79:0/2149348172 conn(0x7f12d40cfc60 msgr2=0x7f12d40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:23.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.121+0000 7f12d8eb6700 1 -- 172.21.15.79:0/2149348172 shutdown_connections 2023-06-23T18:36:23.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.121+0000 7f12d8eb6700 1 -- 172.21.15.79:0/2149348172 wait complete. 2023-06-23T18:36:23.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.122+0000 7f12d8eb6700 1 Processor -- start 2023-06-23T18:36:23.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.122+0000 7f12d8eb6700 1 -- start start 2023-06-23T18:36:23.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.123+0000 7f12d8eb6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12d4158db0 0x7f12d41e6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.123+0000 7f12d8eb6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12d415be70 0x7f12d4150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.123+0000 7f12d8eb6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12d415c420 0x7f12d40b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.123+0000 7f12d8eb6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d41e6780 con 0x7f12d415c420 2023-06-23T18:36:23.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.123+0000 7f12d8eb6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40b1fa0 con 0x7f12d415be70 2023-06-23T18:36:23.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.123+0000 7f12d8eb6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40b2240 con 0x7f12d4158db0 2023-06-23T18:36:23.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.123+0000 7f12d2d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12d415c420 0x7f12d40b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.123+0000 7f12d2d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12d415c420 0x7f12d40b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40166/0 (socket says 172.21.15.79:40166) 2023-06-23T18:36:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.123+0000 7f12d2d9d700 1 -- 172.21.15.79:0/3225853420 learned_addr learned my addr 172.21.15.79:0/3225853420 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.124+0000 7f12d1d9b700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12d415be70 0x7f12d4150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.124+0000 7f12d259c700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12d4158db0 0x7f12d41e6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.124+0000 7f12d2d9d700 1 -- 172.21.15.79:0/3225853420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12d4158db0 msgr2=0x7f12d41e6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.124+0000 7f12d2d9d700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12d4158db0 0x7f12d41e6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.124+0000 7f12d2d9d700 1 -- 172.21.15.79:0/3225853420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12d415be70 msgr2=0x7f12d4150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.124+0000 7f12d2d9d700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12d415be70 0x7f12d4150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.124+0000 7f12d2d9d700 1 -- 172.21.15.79:0/3225853420 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc009a70 con 0x7f12d415c420 2023-06-23T18:36:23.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.124+0000 7f12d1d9b700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12d415be70 0x7f12d4150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:23.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.124+0000 7f12d2d9d700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12d415c420 0x7f12d40b40d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f12d40706d0 tx=0x7f12cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:23.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.124+0000 7f12c2ffd700 1 -- 172.21.15.79:0/3225853420 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc020070 con 0x7f12d415c420 2023-06-23T18:36:23.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.124+0000 7f12c2ffd700 1 -- 172.21.15.79:0/3225853420 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc0098f0 con 0x7f12d415c420 2023-06-23T18:36:23.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.124+0000 7f12d8eb6700 1 -- 172.21.15.79:0/3225853420 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d40b2520 con 0x7f12d415c420 2023-06-23T18:36:23.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.124+0000 7f12c2ffd700 1 -- 172.21.15.79:0/3225853420 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc012430 con 0x7f12d415c420 2023-06-23T18:36:23.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.124+0000 7f12d8eb6700 1 -- 172.21.15.79:0/3225853420 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d40b2ae0 con 0x7f12d415c420 2023-06-23T18:36:23.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.125+0000 7f12d8eb6700 1 -- 172.21.15.79:0/3225853420 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d4066260 con 0x7f12d415c420 2023-06-23T18:36:23.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.125+0000 7f12c2ffd700 1 -- 172.21.15.79:0/3225853420 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f12cc024070 con 0x7f12d415c420 2023-06-23T18:36:23.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.126+0000 7f12c2ffd700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f12b406ace0 0x7f12b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.126+0000 7f12c2ffd700 1 -- 172.21.15.79:0/3225853420 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f12cc08c310 con 0x7f12d415c420 2023-06-23T18:36:23.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.126+0000 7f12d259c700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f12b406ace0 0x7f12b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.126+0000 7f12d259c700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f12b406ace0 0x7f12b406d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f12c8006fd0 tx=0x7f12c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:23.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.130+0000 7f12c2ffd700 1 -- 172.21.15.79:0/3225853420 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f12cc057500 con 0x7f12d415c420 2023-06-23T18:36:23.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.316+0000 7f12d8eb6700 1 -- 172.21.15.79:0/3225853420 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12d40672c0 con 0x7f12d415c420 2023-06-23T18:36:23.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.316+0000 7f12c2ffd700 1 -- 172.21.15.79:0/3225853420 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f12cc05ab20 con 0x7f12d415c420 2023-06-23T18:36:23.321 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:23.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.318+0000 7f12d8eb6700 1 -- 172.21.15.79:0/3225853420 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f12b406ace0 msgr2=0x7f12b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.318+0000 7f12d8eb6700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f12b406ace0 0x7f12b406d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f12c8006fd0 tx=0x7f12c8008040 comp rx=0 tx=0).stop 2023-06-23T18:36:23.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.318+0000 7f12d8eb6700 1 -- 172.21.15.79:0/3225853420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12d415c420 msgr2=0x7f12d40b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.318+0000 7f12d8eb6700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12d415c420 0x7f12d40b40d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f12d40706d0 tx=0x7f12cc004ad0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.318+0000 7f12d8eb6700 1 -- 172.21.15.79:0/3225853420 shutdown_connections 2023-06-23T18:36:23.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.318+0000 7f12d8eb6700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12d415c420 0x7f12d40b40d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.318+0000 7f12d8eb6700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12d4158db0 0x7f12d41e6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.318+0000 7f12d8eb6700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f12b406ace0 0x7f12b406d190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.318+0000 7f12d8eb6700 1 --2- 172.21.15.79:0/3225853420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12d415be70 0x7f12d4150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.318+0000 7f12d8eb6700 1 -- 172.21.15.79:0/3225853420 >> 172.21.15.79:0/3225853420 conn(0x7f12d40cfc60 msgr2=0x7f12d415a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:23.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.318+0000 7f12d8eb6700 1 -- 172.21.15.79:0/3225853420 shutdown_connections 2023-06-23T18:36:23.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.318+0000 7f12d8eb6700 1 -- 172.21.15.79:0/3225853420 wait complete. 2023-06-23T18:36:23.335 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:23.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.449+0000 7f0fe773d700 1 Processor -- start 2023-06-23T18:36:23.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.450+0000 7f0fe773d700 1 -- start start 2023-06-23T18:36:23.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.450+0000 7f0fe773d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0fe00cdad0 0x7f0fe00ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.450+0000 7f0fe773d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe00ce620 0x7f0fe00cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.450+0000 7f0fe773d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe015cd50 con 0x7f0fe0159170 2023-06-23T18:36:23.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.450+0000 7f0fe773d700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe00ceaa0 con 0x7f0fe00cdad0 2023-06-23T18:36:23.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.450+0000 7f0fe773d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe015cfa0 con 0x7f0fe00ce620 2023-06-23T18:36:23.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.450+0000 7f0fe54d9700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f0fe0159170 0x7f0fe00cd3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43212/0 (socket says 172.21.15.79:43212) 2023-06-23T18:36:23.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.450+0000 7f0fe54d9700 1 -- 172.21.15.79:0/3810381319 learned_addr learned my addr 172.21.15.79:0/3810381319 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:23.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fe4cd8700 1 --2- 172.21.15.79:0/3810381319 >> v2:172.21.15.155:3300/0 conn(0x7f0fe00cdad0 0x7f0fe00ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fe5cda700 1 --2- 172.21.15.79:0/3810381319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe00ce620 0x7f0fe00cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 -- 172.21.15.79:0/3810381319 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749161310 0 0) 0x7f0fe015cd50 con 0x7f0fe0159170 2023-06-23T18:36:23.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 -- 172.21.15.79:0/3810381319 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fcc0035b0 con 0x7f0fe0159170 2023-06-23T18:36:23.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 -- 172.21.15.79:0/3810381319 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3509262668 0 0) 0x7f0fcc0035b0 con 0x7f0fe0159170 2023-06-23T18:36:23.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 -- 172.21.15.79:0/3810381319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe00ce620 msgr2=0x7f0fe00cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:23.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 --2- 172.21.15.79:0/3810381319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe00ce620 0x7f0fe00cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 -- 172.21.15.79:0/3810381319 >> v2:172.21.15.155:3300/0 conn(0x7f0fe00cdad0 msgr2=0x7f0fe00ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:23.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 --2- 172.21.15.79:0/3810381319 >> v2:172.21.15.155:3300/0 conn(0x7f0fe00cdad0 0x7f0fe00ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 -- 172.21.15.79:0/3810381319 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe015d280 con 0x7f0fe0159170 2023-06-23T18:36:23.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 -- 172.21.15.79:0/3810381319 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f0fd0001670 con 0x7f0fe0159170 2023-06-23T18:36:23.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 -- 172.21.15.79:0/3810381319 >> v1:172.21.15.79:6789/0 conn(0x7f0fe0159170 legacy=0x7f0fe00cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 --2- 172.21.15.79:0/3810381319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fcc003a00 0x7f0fcc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 --2- 172.21.15.79:0/3810381319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe00ce620 0x7f0fcc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 --2- 172.21.15.79:0/3810381319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fcc004d90 0x7f0fcc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 -- 172.21.15.79:0/3810381319 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe00ceaa0 con 0x7f0fcc004d90 2023-06-23T18:36:23.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 -- 172.21.15.79:0/3810381319 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe015cfa0 con 0x7f0fcc003a00 2023-06-23T18:36:23.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fd7fff700 1 -- 172.21.15.79:0/3810381319 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc00ba10 con 0x7f0fe00ce620 2023-06-23T18:36:23.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fe5cda700 1 --2- 172.21.15.79:0/3810381319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fcc004d90 0x7f0fcc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fe54d9700 1 --2- 172.21.15.79:0/3810381319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fcc003a00 0x7f0fcc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fe4cd8700 1 --2- 172.21.15.79:0/3810381319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe00ce620 0x7f0fcc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.451+0000 7f0fe5cda700 1 -- 172.21.15.79:0/3810381319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe00ce620 msgr2=0x7f0fcc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:23.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fe5cda700 1 --2- 172.21.15.79:0/3810381319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe00ce620 0x7f0fcc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fe5cda700 1 -- 172.21.15.79:0/3810381319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fcc003a00 msgr2=0x7f0fcc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:23.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fe5cda700 1 --2- 172.21.15.79:0/3810381319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fcc003a00 0x7f0fcc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fe5cda700 1 -- 172.21.15.79:0/3810381319 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc0035b0 con 0x7f0fcc004d90 2023-06-23T18:36:23.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fe5cda700 1 --2- 172.21.15.79:0/3810381319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fcc004d90 0x7f0fcc005200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009fc0 tx=0x7f0fcc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97532bde4d7ff095 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fd7fff700 1 -- 172.21.15.79:0/3810381319 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd800ec30 con 0x7f0fcc004d90 2023-06-23T18:36:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fd7fff700 1 -- 172.21.15.79:0/3810381319 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd800f040 con 0x7f0fcc004d90 2023-06-23T18:36:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fd7fff700 1 -- 172.21.15.79:0/3810381319 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8014480 con 0x7f0fcc004d90 2023-06-23T18:36:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fe773d700 1 -- 172.21.15.79:0/3810381319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fcc004d90 msgr2=0x7f0fcc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fe773d700 1 --2- 172.21.15.79:0/3810381319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fcc004d90 0x7f0fcc005200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009fc0 tx=0x7f0fcc005880 comp rx=0 tx=0).stop 2023-06-23T18:36:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fe5cda700 1 -- 172.21.15.79:0/3810381319 reap_dead start 2023-06-23T18:36:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fe773d700 1 -- 172.21.15.79:0/3810381319 shutdown_connections 2023-06-23T18:36:23.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fe773d700 1 -- 172.21.15.79:0/3810381319 >> 172.21.15.79:0/3810381319 conn(0x7f0fe006d660 msgr2=0x7f0fe006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:23.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fe773d700 1 -- 172.21.15.79:0/3810381319 shutdown_connections 2023-06-23T18:36:23.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.452+0000 7f0fe773d700 1 -- 172.21.15.79:0/3810381319 wait complete. 2023-06-23T18:36:23.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.454+0000 7f0fe773d700 1 Processor -- start 2023-06-23T18:36:23.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.454+0000 7f0fe773d700 1 -- start start 2023-06-23T18:36:23.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.454+0000 7f0fe773d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fcc004d90 0x7f0fe00c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.454+0000 7f0fe773d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fe0149bc0 0x7f0fe015b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.454+0000 7f0fe773d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe015beb0 0x7f0fe006a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.454+0000 7f0fe773d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe00c71d0 con 0x7f0fe0149bc0 2023-06-23T18:36:23.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.454+0000 7f0fe773d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe00c68c0 con 0x7f0fe015beb0 2023-06-23T18:36:23.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.454+0000 7f0fe773d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe015c4e0 con 0x7f0fcc004d90 2023-06-23T18:36:23.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.454+0000 7f0fe4cd8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fe0149bc0 0x7f0fe015b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fe4cd8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fe0149bc0 0x7f0fe015b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40184/0 (socket says 172.21.15.79:40184) 2023-06-23T18:36:23.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fe4cd8700 1 -- 172.21.15.79:0/1444993350 learned_addr learned my addr 172.21.15.79:0/1444993350 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:23.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fe5cda700 1 --2- 172.21.15.79:0/1444993350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe015beb0 0x7f0fe006a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fe54d9700 1 --2- 172.21.15.79:0/1444993350 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fcc004d90 0x7f0fe00c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fe4cd8700 1 -- 172.21.15.79:0/1444993350 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fcc004d90 msgr2=0x7f0fe00c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fe4cd8700 1 --2- 172.21.15.79:0/1444993350 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fcc004d90 0x7f0fe00c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fe4cd8700 1 -- 172.21.15.79:0/1444993350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe015beb0 msgr2=0x7f0fe006a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fe4cd8700 1 --2- 172.21.15.79:0/1444993350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe015beb0 0x7f0fe006a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fe4cd8700 1 -- 172.21.15.79:0/1444993350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe00c6b10 con 0x7f0fe0149bc0 2023-06-23T18:36:23.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fe4cd8700 1 --2- 172.21.15.79:0/1444993350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fe0149bc0 0x7f0fe015b970 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc011aa0 tx=0x7f0fdc011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:23.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fd5ffb700 1 -- 172.21.15.79:0/1444993350 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc011ee0 con 0x7f0fe0149bc0 2023-06-23T18:36:23.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fd5ffb700 1 -- 172.21.15.79:0/1444993350 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc00f940 con 0x7f0fe0149bc0 2023-06-23T18:36:23.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fe773d700 1 -- 172.21.15.79:0/1444993350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe00c6df0 con 0x7f0fe0149bc0 2023-06-23T18:36:23.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fd5ffb700 1 -- 172.21.15.79:0/1444993350 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc00fcf0 con 0x7f0fe0149bc0 2023-06-23T18:36:23.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.455+0000 7f0fe773d700 1 -- 172.21.15.79:0/1444993350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe014a030 con 0x7f0fe0149bc0 2023-06-23T18:36:23.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.456+0000 7f0fe773d700 1 -- 172.21.15.79:0/1444993350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe015c280 con 0x7f0fe0149bc0 2023-06-23T18:36:23.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.457+0000 7f0fd5ffb700 1 -- 172.21.15.79:0/1444993350 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0fdc01d400 con 0x7f0fe0149bc0 2023-06-23T18:36:23.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.457+0000 7f0fd5ffb700 1 --2- 172.21.15.79:0/1444993350 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0fc406ace0 0x7f0fc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.457+0000 7f0fd5ffb700 1 -- 172.21.15.79:0/1444993350 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0fdc089af0 con 0x7f0fe0149bc0 2023-06-23T18:36:23.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.457+0000 7f0fe54d9700 1 --2- 172.21.15.79:0/1444993350 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0fc406ace0 0x7f0fc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.457+0000 7f0fe54d9700 1 --2- 172.21.15.79:0/1444993350 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0fc406ace0 0x7f0fc406d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0001cf0 tx=0x7f0fd0007980 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:23.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.461+0000 7f0fd5ffb700 1 -- 172.21.15.79:0/1444993350 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0fdc054ba0 con 0x7f0fe0149bc0 2023-06-23T18:36:23.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.647+0000 7f0fe773d700 1 -- 172.21.15.79:0/1444993350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fe015d530 con 0x7f0fe0149bc0 2023-06-23T18:36:23.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.649+0000 7f0fd5ffb700 1 -- 172.21.15.79:0/1444993350 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7f0fdc0581c0 con 0x7f0fe0149bc0 2023-06-23T18:36:23.654 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:23.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.651+0000 7f0fe773d700 1 -- 172.21.15.79:0/1444993350 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0fc406ace0 msgr2=0x7f0fc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.651+0000 7f0fe773d700 1 --2- 172.21.15.79:0/1444993350 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0fc406ace0 0x7f0fc406d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0001cf0 tx=0x7f0fd0007980 comp rx=0 tx=0).stop 2023-06-23T18:36:23.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.651+0000 7f0fe773d700 1 -- 172.21.15.79:0/1444993350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fe0149bc0 msgr2=0x7f0fe015b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.651+0000 7f0fe773d700 1 --2- 172.21.15.79:0/1444993350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fe0149bc0 0x7f0fe015b970 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc011aa0 tx=0x7f0fdc011db0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.651+0000 7f0fe773d700 1 -- 172.21.15.79:0/1444993350 shutdown_connections 2023-06-23T18:36:23.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.651+0000 7f0fe773d700 1 --2- 172.21.15.79:0/1444993350 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0fc406ace0 0x7f0fc406d190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.651+0000 7f0fe773d700 1 --2- 172.21.15.79:0/1444993350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe015beb0 0x7f0fe006a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.651+0000 7f0fe773d700 1 --2- 172.21.15.79:0/1444993350 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fcc004d90 0x7f0fe00c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.651+0000 7f0fe773d700 1 --2- 172.21.15.79:0/1444993350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fe0149bc0 0x7f0fe015b970 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.651+0000 7f0fe773d700 1 -- 172.21.15.79:0/1444993350 >> 172.21.15.79:0/1444993350 conn(0x7f0fe006d660 msgr2=0x7f0fe00ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:23.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.651+0000 7f0fe773d700 1 -- 172.21.15.79:0/1444993350 shutdown_connections 2023-06-23T18:36:23.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.651+0000 7f0fe773d700 1 -- 172.21.15.79:0/1444993350 wait complete. 2023-06-23T18:36:23.668 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:23.669 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:23.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.786+0000 7f6374bfe700 1 Processor -- start 2023-06-23T18:36:23.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.786+0000 7f6374bfe700 1 -- start start 2023-06-23T18:36:23.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f6374bfe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f637015c2a0 0x7f63700ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f6374bfe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f637006a5a0 0x7f63700c9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f6374bfe700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637015ffe0 con 0x7f63700b1410 2023-06-23T18:36:23.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f6374bfe700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63701602c0 con 0x7f637006a5a0 2023-06-23T18:36:23.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f6374bfe700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63701605a0 con 0x7f637015c2a0 2023-06-23T18:36:23.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f636ed9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f637006a5a0 0x7f63700c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f636dd9b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f63700b1410 0x7f6370069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43236/0 (socket says 172.21.15.79:43236) 2023-06-23T18:36:23.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f636ed9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f637006a5a0 0x7f63700c9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40192/0 (socket says 172.21.15.79:40192) 2023-06-23T18:36:23.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f636dd9b700 1 -- 172.21.15.79:0/2990531904 learned_addr learned my addr 172.21.15.79:0/2990531904 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:23.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f636e59c700 1 --2- 172.21.15.79:0/2990531904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f637015c2a0 0x7f63700ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f636ed9d700 1 -- 172.21.15.79:0/2990531904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f637015c2a0 msgr2=0x7f63700ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:23.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f636ed9d700 1 --2- 172.21.15.79:0/2990531904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f637015c2a0 0x7f63700ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f636ed9d700 1 -- 172.21.15.79:0/2990531904 >> v1:172.21.15.79:6789/0 conn(0x7f63700b1410 legacy=0x7f6370069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f636ed9d700 1 -- 172.21.15.79:0/2990531904 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6370160880 con 0x7f637006a5a0 2023-06-23T18:36:23.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f636d59a700 1 -- 172.21.15.79:0/2990531904 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082350122 0 0) 0x7f637015ffe0 con 0x7f63700b1410 2023-06-23T18:36:23.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f636e59c700 1 --2- 172.21.15.79:0/2990531904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f637015c2a0 0x7f63700ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:23.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.787+0000 7f636ed9d700 1 --2- 172.21.15.79:0/2990531904 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f637006a5a0 0x7f63700c9720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6368009d90 tx=0x7f636800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8f3d1d4bb021d4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:23.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.788+0000 7f636d59a700 1 -- 172.21.15.79:0/2990531904 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636800e070 con 0x7f637006a5a0 2023-06-23T18:36:23.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.788+0000 7f636d59a700 1 -- 172.21.15.79:0/2990531904 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636800bbd0 con 0x7f637006a5a0 2023-06-23T18:36:23.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.788+0000 7f636d59a700 1 -- 172.21.15.79:0/2990531904 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368012430 con 0x7f637006a5a0 2023-06-23T18:36:23.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.788+0000 7f6374bfe700 1 -- 172.21.15.79:0/2990531904 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f637006a5a0 msgr2=0x7f63700c9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.788+0000 7f6374bfe700 1 --2- 172.21.15.79:0/2990531904 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f637006a5a0 0x7f63700c9720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6368009d90 tx=0x7f636800b330 comp rx=0 tx=0).stop 2023-06-23T18:36:23.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.788+0000 7f6374bfe700 1 -- 172.21.15.79:0/2990531904 shutdown_connections 2023-06-23T18:36:23.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.788+0000 7f6374bfe700 1 --2- 172.21.15.79:0/2990531904 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f637006a5a0 0x7f63700c9720 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.788+0000 7f6374bfe700 1 --2- 172.21.15.79:0/2990531904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f637015c2a0 0x7f63700ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.788+0000 7f6374bfe700 1 -- 172.21.15.79:0/2990531904 >> 172.21.15.79:0/2990531904 conn(0x7f63700cfc60 msgr2=0x7f63700b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:23.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.788+0000 7f6374bfe700 1 -- 172.21.15.79:0/2990531904 shutdown_connections 2023-06-23T18:36:23.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.788+0000 7f6374bfe700 1 -- 172.21.15.79:0/2990531904 wait complete. 2023-06-23T18:36:23.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.789+0000 7f6374bfe700 1 Processor -- start 2023-06-23T18:36:23.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.790+0000 7f6374bfe700 1 -- start start 2023-06-23T18:36:23.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.790+0000 7f6374bfe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f637006a5a0 0x7f6370153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.790+0000 7f6374bfe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63700b1410 0x7f63700c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.790+0000 7f6374bfe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f637015c2a0 0x7f63700c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.790+0000 7f6374bfe700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63700c6d50 con 0x7f63700b1410 2023-06-23T18:36:23.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.790+0000 7f6374bfe700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63700c6140 con 0x7f637015c2a0 2023-06-23T18:36:23.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.790+0000 7f6374bfe700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63700c6420 con 0x7f637006a5a0 2023-06-23T18:36:23.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.790+0000 7f636dd9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63700b1410 0x7f63700c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.790+0000 7f636dd9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63700b1410 0x7f63700c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40198/0 (socket says 172.21.15.79:40198) 2023-06-23T18:36:23.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.790+0000 7f636dd9b700 1 -- 172.21.15.79:0/338580412 learned_addr learned my addr 172.21.15.79:0/338580412 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:23.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.790+0000 7f636e59c700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f637006a5a0 0x7f6370153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.790+0000 7f636ed9d700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f637015c2a0 0x7f63700c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.791+0000 7f636dd9b700 1 -- 172.21.15.79:0/338580412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f637006a5a0 msgr2=0x7f6370153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.791+0000 7f636dd9b700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f637006a5a0 0x7f6370153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.791+0000 7f636dd9b700 1 -- 172.21.15.79:0/338580412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f637015c2a0 msgr2=0x7f63700c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.791+0000 7f636dd9b700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f637015c2a0 0x7f63700c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.791+0000 7f636dd9b700 1 -- 172.21.15.79:0/338580412 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6368009a70 con 0x7f63700b1410 2023-06-23T18:36:23.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.791+0000 7f636ed9d700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f637015c2a0 0x7f63700c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:23.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.791+0000 7f636dd9b700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63700b1410 0x7f63700c71e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f636400c550 tx=0x7f636400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:23.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.791+0000 7f635effd700 1 -- 172.21.15.79:0/338580412 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6364004c80 con 0x7f63700b1410 2023-06-23T18:36:23.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.791+0000 7f635effd700 1 -- 172.21.15.79:0/338580412 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6364004e00 con 0x7f63700b1410 2023-06-23T18:36:23.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.791+0000 7f635effd700 1 -- 172.21.15.79:0/338580412 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6364007310 con 0x7f63700b1410 2023-06-23T18:36:23.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.791+0000 7f6374bfe700 1 -- 172.21.15.79:0/338580412 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63700c6700 con 0x7f63700b1410 2023-06-23T18:36:23.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.791+0000 7f6374bfe700 1 -- 172.21.15.79:0/338580412 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6370153d50 con 0x7f63700b1410 2023-06-23T18:36:23.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.792+0000 7f6374bfe700 1 -- 172.21.15.79:0/338580412 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63700c6f00 con 0x7f63700b1410 2023-06-23T18:36:23.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.792+0000 7f635effd700 1 -- 172.21.15.79:0/338580412 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6364007470 con 0x7f63700b1410 2023-06-23T18:36:23.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.793+0000 7f635effd700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f635006ace0 0x7f635006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:23.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.793+0000 7f635effd700 1 -- 172.21.15.79:0/338580412 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f63640890c0 con 0x7f63700b1410 2023-06-23T18:36:23.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.793+0000 7f636e59c700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f635006ace0 0x7f635006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:23.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.793+0000 7f636e59c700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f635006ace0 0x7f635006d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6360006fd0 tx=0x7f6360008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:23.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.797+0000 7f635effd700 1 -- 172.21.15.79:0/338580412 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f63700c6f00 con 0x7f63700b1410 2023-06-23T18:36:23.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.982+0000 7f6374bfe700 1 -- 172.21.15.79:0/338580412 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f63700672c0 con 0x7f63700b1410 2023-06-23T18:36:23.987 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.982+0000 7f635effd700 1 -- 172.21.15.79:0/338580412 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6364054200 con 0x7f63700b1410 2023-06-23T18:36:23.987 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:23.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.984+0000 7f6374bfe700 1 -- 172.21.15.79:0/338580412 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f635006ace0 msgr2=0x7f635006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.984+0000 7f6374bfe700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f635006ace0 0x7f635006d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6360006fd0 tx=0x7f6360008040 comp rx=0 tx=0).stop 2023-06-23T18:36:23.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.984+0000 7f6374bfe700 1 -- 172.21.15.79:0/338580412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63700b1410 msgr2=0x7f63700c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:23.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.984+0000 7f6374bfe700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63700b1410 0x7f63700c71e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f636400c550 tx=0x7f636400c910 comp rx=0 tx=0).stop 2023-06-23T18:36:23.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.984+0000 7f6374bfe700 1 -- 172.21.15.79:0/338580412 shutdown_connections 2023-06-23T18:36:23.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.984+0000 7f6374bfe700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f635006ace0 0x7f635006d190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.984+0000 7f6374bfe700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f637015c2a0 0x7f63700c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.984+0000 7f6374bfe700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f637006a5a0 0x7f6370153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.984+0000 7f6374bfe700 1 --2- 172.21.15.79:0/338580412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63700b1410 0x7f63700c71e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:23.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.984+0000 7f6374bfe700 1 -- 172.21.15.79:0/338580412 >> 172.21.15.79:0/338580412 conn(0x7f63700cfc60 msgr2=0x7f637006ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:23.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.984+0000 7f6374bfe700 1 -- 172.21.15.79:0/338580412 shutdown_connections 2023-06-23T18:36:23.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:23.984+0000 7f6374bfe700 1 -- 172.21.15.79:0/338580412 wait complete. 2023-06-23T18:36:24.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:24.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.119+0000 7febe8572700 1 Processor -- start 2023-06-23T18:36:24.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.120+0000 7febe8572700 1 -- start start 2023-06-23T18:36:24.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.120+0000 7febe8572700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7febe00cd440 0x7febe01608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.120+0000 7febe8572700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe00cdc90 con 0x7febe006a5a0 2023-06-23T18:36:24.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.120+0000 7febe8572700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe015d0f0 con 0x7febe01569c0 2023-06-23T18:36:24.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.120+0000 7febe8572700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe00cf570 con 0x7febe00cd440 2023-06-23T18:36:24.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.121+0000 7febe6b0f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7febe00cd440 0x7febe01608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.121+0000 7febe5b0d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7febe006a5a0 0x7febe00b12f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43248/0 (socket says 172.21.15.79:43248) 2023-06-23T18:36:24.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.121+0000 7febe5b0d700 1 -- 172.21.15.79:0/2565680815 learned_addr learned my addr 172.21.15.79:0/2565680815 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:24.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.121+0000 7febe6b0f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7febe00cd440 0x7febe01608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40208/0 (socket says 172.21.15.79:40208) 2023-06-23T18:36:24.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.121+0000 7febe6b0f700 1 -- 172.21.15.79:0/2565680815 >> v1:172.21.15.155:6789/0 conn(0x7febe01569c0 legacy=0x7febe0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:24.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.121+0000 7febe6b0f700 1 -- 172.21.15.79:0/2565680815 >> v1:172.21.15.79:6789/0 conn(0x7febe006a5a0 legacy=0x7febe00b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.121+0000 7febe6b0f700 1 -- 172.21.15.79:0/2565680815 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe0156d90 con 0x7febe00cd440 2023-06-23T18:36:24.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.121+0000 7febe6b0f700 1 --2- 172.21.15.79:0/2565680815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe00cd440 0x7febe01608b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7febd8009d90 tx=0x7febd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639ba68df13a3bd1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:24.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.121+0000 7febe530c700 1 -- 172.21.15.79:0/2565680815 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd800e070 con 0x7febe00cd440 2023-06-23T18:36:24.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.121+0000 7febe530c700 1 -- 172.21.15.79:0/2565680815 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd800bbd0 con 0x7febe00cd440 2023-06-23T18:36:24.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.121+0000 7febe530c700 1 -- 172.21.15.79:0/2565680815 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd8003a20 con 0x7febe00cd440 2023-06-23T18:36:24.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.122+0000 7febe8572700 1 -- 172.21.15.79:0/2565680815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe00cd440 msgr2=0x7febe01608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.122+0000 7febe8572700 1 --2- 172.21.15.79:0/2565680815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe00cd440 0x7febe01608b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7febd8009d90 tx=0x7febd800b330 comp rx=0 tx=0).stop 2023-06-23T18:36:24.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.122+0000 7febe8572700 1 -- 172.21.15.79:0/2565680815 shutdown_connections 2023-06-23T18:36:24.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.122+0000 7febe8572700 1 --2- 172.21.15.79:0/2565680815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe00cd440 0x7febe01608b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.122+0000 7febe8572700 1 -- 172.21.15.79:0/2565680815 >> 172.21.15.79:0/2565680815 conn(0x7febe006afa0 msgr2=0x7febe006b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:24.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.122+0000 7febe8572700 1 -- 172.21.15.79:0/2565680815 shutdown_connections 2023-06-23T18:36:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.122+0000 7febe8572700 1 -- 172.21.15.79:0/2565680815 wait complete. 2023-06-23T18:36:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.123+0000 7febe8572700 1 Processor -- start 2023-06-23T18:36:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.123+0000 7febe8572700 1 -- start start 2023-06-23T18:36:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe8572700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe006a5a0 0x7febe00c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe8572700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febe00cd440 0x7febe00b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe8572700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febe01569c0 0x7febe00b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe8572700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe01501a0 con 0x7febe006a5a0 2023-06-23T18:36:24.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe8572700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe00b5550 con 0x7febe01569c0 2023-06-23T18:36:24.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe8572700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe00c6510 con 0x7febe00cd440 2023-06-23T18:36:24.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe630e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe006a5a0 0x7febe00c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe630e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe006a5a0 0x7febe00c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40210/0 (socket says 172.21.15.79:40210) 2023-06-23T18:36:24.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe630e700 1 -- 172.21.15.79:0/1194069408 learned_addr learned my addr 172.21.15.79:0/1194069408 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:24.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe630e700 1 -- 172.21.15.79:0/1194069408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febe00cd440 msgr2=0x7febe00b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe5b0d700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febe00cd440 0x7febe00b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe6b0f700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febe01569c0 0x7febe00b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe630e700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febe00cd440 0x7febe00b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe630e700 1 -- 172.21.15.79:0/1194069408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febe01569c0 msgr2=0x7febe00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe630e700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febe01569c0 0x7febe00b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe630e700 1 -- 172.21.15.79:0/1194069408 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd8009a70 con 0x7febe006a5a0 2023-06-23T18:36:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe5b0d700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febe00cd440 0x7febe00b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe6b0f700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febe01569c0 0x7febe00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe630e700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe006a5a0 0x7febe00c5fd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febd000b4a0 tx=0x7febd000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febd6ffd700 1 -- 172.21.15.79:0/1194069408 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0011830 con 0x7febe006a5a0 2023-06-23T18:36:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febd6ffd700 1 -- 172.21.15.79:0/1194069408 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd0011990 con 0x7febe006a5a0 2023-06-23T18:36:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe8572700 1 -- 172.21.15.79:0/1194069408 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe00b31e0 con 0x7febe006a5a0 2023-06-23T18:36:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febd6ffd700 1 -- 172.21.15.79:0/1194069408 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0011d40 con 0x7febe006a5a0 2023-06-23T18:36:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.124+0000 7febe8572700 1 -- 172.21.15.79:0/1194069408 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe00b3480 con 0x7febe006a5a0 2023-06-23T18:36:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.125+0000 7febe8572700 1 -- 172.21.15.79:0/1194069408 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe00672c0 con 0x7febe006a5a0 2023-06-23T18:36:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.126+0000 7febd6ffd700 1 -- 172.21.15.79:0/1194069408 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7febd0004020 con 0x7febe006a5a0 2023-06-23T18:36:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.126+0000 7febd6ffd700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7febc406ace0 0x7febc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.126+0000 7febd6ffd700 1 -- 172.21.15.79:0/1194069408 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7febd0089590 con 0x7febe006a5a0 2023-06-23T18:36:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.127+0000 7febe5b0d700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7febc406ace0 0x7febc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.127+0000 7febe5b0d700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7febc406ace0 0x7febc406d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7febe00cbb90 tx=0x7febdc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.130+0000 7febd6ffd700 1 -- 172.21.15.79:0/1194069408 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7febd00547b0 con 0x7febe006a5a0 2023-06-23T18:36:24.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.316+0000 7febe8572700 1 -- 172.21.15.79:0/1194069408 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7febe00ccb70 con 0x7febe006a5a0 2023-06-23T18:36:24.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.316+0000 7febd6ffd700 1 -- 172.21.15.79:0/1194069408 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7febd0057dd0 con 0x7febe006a5a0 2023-06-23T18:36:24.322 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:24.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.318+0000 7febe8572700 1 -- 172.21.15.79:0/1194069408 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7febc406ace0 msgr2=0x7febc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.318+0000 7febe8572700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7febc406ace0 0x7febc406d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7febe00cbb90 tx=0x7febdc009040 comp rx=0 tx=0).stop 2023-06-23T18:36:24.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.318+0000 7febe8572700 1 -- 172.21.15.79:0/1194069408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe006a5a0 msgr2=0x7febe00c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.318+0000 7febe8572700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe006a5a0 0x7febe00c5fd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febd000b4a0 tx=0x7febd000b860 comp rx=0 tx=0).stop 2023-06-23T18:36:24.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.318+0000 7febe8572700 1 -- 172.21.15.79:0/1194069408 shutdown_connections 2023-06-23T18:36:24.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.318+0000 7febe8572700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7febc406ace0 0x7febc406d190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.318+0000 7febe8572700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febe01569c0 0x7febe00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.318+0000 7febe8572700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe006a5a0 0x7febe00c5fd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.318+0000 7febe8572700 1 --2- 172.21.15.79:0/1194069408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febe00cd440 0x7febe00b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.318+0000 7febe8572700 1 -- 172.21.15.79:0/1194069408 >> 172.21.15.79:0/1194069408 conn(0x7febe006afa0 msgr2=0x7febe015f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:24.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.318+0000 7febe8572700 1 -- 172.21.15.79:0/1194069408 shutdown_connections 2023-06-23T18:36:24.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.318+0000 7febe8572700 1 -- 172.21.15.79:0/1194069408 wait complete. 2023-06-23T18:36:24.335 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:24.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.452+0000 7f654290d700 1 Processor -- start 2023-06-23T18:36:24.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.452+0000 7f654290d700 1 -- start start 2023-06-23T18:36:24.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.453+0000 7f654290d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f653c156ae0 0x7f653c15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.453+0000 7f654290d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f653c0cc4b0 0x7f653c071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.453+0000 7f654290d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f653c15daf0 con 0x7f653c0cc060 2023-06-23T18:36:24.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.453+0000 7f654290d700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653c15a560 con 0x7f653c156ae0 2023-06-23T18:36:24.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.453+0000 7f654290d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c159950 con 0x7f653c0cc4b0 2023-06-23T18:36:24.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.453+0000 7f653bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f653c156ae0 0x7f653c15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.454+0000 7f653bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f653c156ae0 0x7f653c15d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40218/0 (socket says 172.21.15.79:40218) 2023-06-23T18:36:24.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.454+0000 7f653bfff700 1 -- 172.21.15.79:0/1846119834 learned_addr learned my addr 172.21.15.79:0/1846119834 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:24.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.454+0000 7f6540eaa700 1 --2- 172.21.15.79:0/1846119834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f653c0cc4b0 0x7f653c071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.454+0000 7f653bfff700 1 -- 172.21.15.79:0/1846119834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f653c0cc4b0 msgr2=0x7f653c071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:24.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.454+0000 7f653bfff700 1 --2- 172.21.15.79:0/1846119834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f653c0cc4b0 0x7f653c071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.454+0000 7f653bfff700 1 -- 172.21.15.79:0/1846119834 >> v1:172.21.15.155:6789/0 conn(0x7f653c0cc060 legacy=0x7f653c071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:24.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.454+0000 7f653bfff700 1 -- 172.21.15.79:0/1846119834 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c159c30 con 0x7f653c156ae0 2023-06-23T18:36:24.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.454+0000 7f653bfff700 1 --2- 172.21.15.79:0/1846119834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f653c156ae0 0x7f653c15d5b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f652c00bb90 tx=0x7f652c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e476ad54c518985 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:24.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.454+0000 7f653affd700 1 -- 172.21.15.79:0/1846119834 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c00dd10 con 0x7f653c156ae0 2023-06-23T18:36:24.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.454+0000 7f653affd700 1 -- 172.21.15.79:0/1846119834 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652c00de70 con 0x7f653c156ae0 2023-06-23T18:36:24.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.454+0000 7f653affd700 1 -- 172.21.15.79:0/1846119834 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c004230 con 0x7f653c156ae0 2023-06-23T18:36:24.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.455+0000 7f654290d700 1 -- 172.21.15.79:0/1846119834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f653c156ae0 msgr2=0x7f653c15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.455+0000 7f654290d700 1 --2- 172.21.15.79:0/1846119834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f653c156ae0 0x7f653c15d5b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f652c00bb90 tx=0x7f652c00bea0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.455+0000 7f654290d700 1 -- 172.21.15.79:0/1846119834 shutdown_connections 2023-06-23T18:36:24.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.455+0000 7f654290d700 1 --2- 172.21.15.79:0/1846119834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f653c156ae0 0x7f653c15d5b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.455+0000 7f654290d700 1 --2- 172.21.15.79:0/1846119834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f653c0cc4b0 0x7f653c071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.455+0000 7f654290d700 1 -- 172.21.15.79:0/1846119834 >> 172.21.15.79:0/1846119834 conn(0x7f653c06afa0 msgr2=0x7f653c06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:24.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.455+0000 7f654290d700 1 -- 172.21.15.79:0/1846119834 shutdown_connections 2023-06-23T18:36:24.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.455+0000 7f654290d700 1 -- 172.21.15.79:0/1846119834 wait complete. 2023-06-23T18:36:24.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.456+0000 7f654290d700 1 Processor -- start 2023-06-23T18:36:24.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.456+0000 7f654290d700 1 -- start start 2023-06-23T18:36:24.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f654290d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f653c0cc060 0x7f653c148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f654290d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f653c0cc4b0 0x7f653c0b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f654290d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f653c156ae0 0x7f653c157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f654290d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c0b3d70 con 0x7f653c156ae0 2023-06-23T18:36:24.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f654290d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c0b3160 con 0x7f653c0cc060 2023-06-23T18:36:24.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f654290d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c0b3440 con 0x7f653c0cc4b0 2023-06-23T18:36:24.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f6540eaa700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f653c156ae0 0x7f653c157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f6540eaa700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f653c156ae0 0x7f653c157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40228/0 (socket says 172.21.15.79:40228) 2023-06-23T18:36:24.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f6540eaa700 1 -- 172.21.15.79:0/3010206742 learned_addr learned my addr 172.21.15.79:0/3010206742 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:24.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f653b7fe700 1 --2- 172.21.15.79:0/3010206742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f653c0cc4b0 0x7f653c0b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f653bfff700 1 --2- 172.21.15.79:0/3010206742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f653c0cc060 0x7f653c148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f6540eaa700 1 -- 172.21.15.79:0/3010206742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f653c0cc4b0 msgr2=0x7f653c0b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f6540eaa700 1 --2- 172.21.15.79:0/3010206742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f653c0cc4b0 0x7f653c0b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f6540eaa700 1 -- 172.21.15.79:0/3010206742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f653c0cc060 msgr2=0x7f653c148a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f6540eaa700 1 --2- 172.21.15.79:0/3010206742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f653c0cc060 0x7f653c148a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f6540eaa700 1 -- 172.21.15.79:0/3010206742 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652c00b7f0 con 0x7f653c156ae0 2023-06-23T18:36:24.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f6540eaa700 1 --2- 172.21.15.79:0/3010206742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f653c156ae0 0x7f653c157f00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f653400cd70 tx=0x7f653400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:24.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f6538ff9700 1 -- 172.21.15.79:0/3010206742 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534003a20 con 0x7f653c156ae0 2023-06-23T18:36:24.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f6538ff9700 1 -- 172.21.15.79:0/3010206742 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6534003b80 con 0x7f653c156ae0 2023-06-23T18:36:24.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f654290d700 1 -- 172.21.15.79:0/3010206742 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653c0b3720 con 0x7f653c156ae0 2023-06-23T18:36:24.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.457+0000 7f6538ff9700 1 -- 172.21.15.79:0/3010206742 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534004020 con 0x7f653c156ae0 2023-06-23T18:36:24.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.458+0000 7f654290d700 1 -- 172.21.15.79:0/3010206742 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c153d30 con 0x7f653c156ae0 2023-06-23T18:36:24.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.458+0000 7f654290d700 1 -- 172.21.15.79:0/3010206742 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653c0672c0 con 0x7f653c156ae0 2023-06-23T18:36:24.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.459+0000 7f6538ff9700 1 -- 172.21.15.79:0/3010206742 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6534004180 con 0x7f653c156ae0 2023-06-23T18:36:24.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.459+0000 7f6538ff9700 1 --2- 172.21.15.79:0/3010206742 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f652806ad10 0x7f652806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.459+0000 7f6538ff9700 1 -- 172.21.15.79:0/3010206742 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6534012070 con 0x7f653c156ae0 2023-06-23T18:36:24.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.459+0000 7f653bfff700 1 --2- 172.21.15.79:0/3010206742 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f652806ad10 0x7f652806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.460+0000 7f653bfff700 1 --2- 172.21.15.79:0/3010206742 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f652806ad10 0x7f652806d1c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f652c000c00 tx=0x7f652c013000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:24.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.463+0000 7f6538ff9700 1 -- 172.21.15.79:0/3010206742 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6534054d00 con 0x7f653c156ae0 2023-06-23T18:36:24.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.649+0000 7f654290d700 1 -- 172.21.15.79:0/3010206742 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f653c1595a0 con 0x7f653c156ae0 2023-06-23T18:36:24.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.649+0000 7f6538ff9700 1 -- 172.21.15.79:0/3010206742 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6534058320 con 0x7f653c156ae0 2023-06-23T18:36:24.655 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:24.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.652+0000 7f654290d700 1 -- 172.21.15.79:0/3010206742 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f652806ad10 msgr2=0x7f652806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.652+0000 7f654290d700 1 --2- 172.21.15.79:0/3010206742 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f652806ad10 0x7f652806d1c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f652c000c00 tx=0x7f652c013000 comp rx=0 tx=0).stop 2023-06-23T18:36:24.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.652+0000 7f654290d700 1 -- 172.21.15.79:0/3010206742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f653c156ae0 msgr2=0x7f653c157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.652+0000 7f654290d700 1 --2- 172.21.15.79:0/3010206742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f653c156ae0 0x7f653c157f00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f653400cd70 tx=0x7f653400a570 comp rx=0 tx=0).stop 2023-06-23T18:36:24.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.652+0000 7f654290d700 1 -- 172.21.15.79:0/3010206742 shutdown_connections 2023-06-23T18:36:24.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.652+0000 7f654290d700 1 --2- 172.21.15.79:0/3010206742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f653c156ae0 0x7f653c157f00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.652+0000 7f654290d700 1 --2- 172.21.15.79:0/3010206742 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f652806ad10 0x7f652806d1c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.652+0000 7f654290d700 1 --2- 172.21.15.79:0/3010206742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f653c0cc060 0x7f653c148a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.652+0000 7f654290d700 1 --2- 172.21.15.79:0/3010206742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f653c0cc4b0 0x7f653c0b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.652+0000 7f654290d700 1 -- 172.21.15.79:0/3010206742 >> 172.21.15.79:0/3010206742 conn(0x7f653c06afa0 msgr2=0x7f653c0c5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:24.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.652+0000 7f654290d700 1 -- 172.21.15.79:0/3010206742 shutdown_connections 2023-06-23T18:36:24.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.652+0000 7f654290d700 1 -- 172.21.15.79:0/3010206742 wait complete. 2023-06-23T18:36:24.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:24.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.786+0000 7f0f7427b700 1 Processor -- start 2023-06-23T18:36:24.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.786+0000 7f0f7427b700 1 -- start start 2023-06-23T18:36:24.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f7427b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0f6c1438f0 0x7f0f6c0b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f7427b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0f6c0d08c0 0x7f0f6c0d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f7427b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f6c0d1200 con 0x7f0f6c0cedf0 2023-06-23T18:36:24.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f7427b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6c0b0b10 con 0x7f0f6c1438f0 2023-06-23T18:36:24.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f7427b700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6c0b0df0 con 0x7f0f6c0d08c0 2023-06-23T18:36:24.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f71816700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0f6c1438f0 0x7f0f6c0b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f71816700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0f6c1438f0 0x7f0f6c0b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40236/0 (socket says 172.21.15.79:40236) 2023-06-23T18:36:24.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f71816700 1 -- 172.21.15.79:0/1035190068 learned_addr learned my addr 172.21.15.79:0/1035190068 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:24.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f72818700 1 --2- 172.21.15.79:0/1035190068 >> v2:172.21.15.155:3300/0 conn(0x7f0f6c0d08c0 0x7f0f6c0d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f71816700 1 -- 172.21.15.79:0/1035190068 >> v2:172.21.15.155:3300/0 conn(0x7f0f6c0d08c0 msgr2=0x7f0f6c0d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:24.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f71816700 1 --2- 172.21.15.79:0/1035190068 >> v2:172.21.15.155:3300/0 conn(0x7f0f6c0d08c0 0x7f0f6c0d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f71816700 1 -- 172.21.15.79:0/1035190068 >> v1:172.21.15.155:6789/0 conn(0x7f0f6c0cedf0 legacy=0x7f0f6c0b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:24.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f71816700 1 -- 172.21.15.79:0/1035190068 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c0b10d0 con 0x7f0f6c1438f0 2023-06-23T18:36:24.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f71816700 1 --2- 172.21.15.79:0/1035190068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f6c1438f0 0x7f0f6c0b1bb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0668b0 tx=0x7f0f68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc7b50aa15fbae7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:24.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.787+0000 7f0f71015700 1 -- 172.21.15.79:0/1035190068 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68004230 con 0x7f0f6c1438f0 2023-06-23T18:36:24.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.788+0000 7f0f71015700 1 -- 172.21.15.79:0/1035190068 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f68004390 con 0x7f0f6c1438f0 2023-06-23T18:36:24.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.788+0000 7f0f71015700 1 -- 172.21.15.79:0/1035190068 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680036a0 con 0x7f0f6c1438f0 2023-06-23T18:36:24.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.788+0000 7f0f7427b700 1 -- 172.21.15.79:0/1035190068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f6c1438f0 msgr2=0x7f0f6c0b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.788+0000 7f0f7427b700 1 --2- 172.21.15.79:0/1035190068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f6c1438f0 0x7f0f6c0b1bb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0668b0 tx=0x7f0f68009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.788+0000 7f0f7427b700 1 -- 172.21.15.79:0/1035190068 shutdown_connections 2023-06-23T18:36:24.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.788+0000 7f0f7427b700 1 --2- 172.21.15.79:0/1035190068 >> v2:172.21.15.155:3300/0 conn(0x7f0f6c0d08c0 0x7f0f6c0d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.788+0000 7f0f7427b700 1 --2- 172.21.15.79:0/1035190068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f6c1438f0 0x7f0f6c0b1bb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.788+0000 7f0f7427b700 1 -- 172.21.15.79:0/1035190068 >> 172.21.15.79:0/1035190068 conn(0x7f0f6c06a7b0 msgr2=0x7f0f6c099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:24.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.788+0000 7f0f7427b700 1 -- 172.21.15.79:0/1035190068 shutdown_connections 2023-06-23T18:36:24.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.788+0000 7f0f7427b700 1 -- 172.21.15.79:0/1035190068 wait complete. 2023-06-23T18:36:24.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.790+0000 7f0f7427b700 1 Processor -- start 2023-06-23T18:36:24.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.790+0000 7f0f7427b700 1 -- start start 2023-06-23T18:36:24.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f7427b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f6c0cedf0 0x7f0f6c15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f7427b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f6c0d08c0 0x7f0f6c09a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f7427b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f6c1438f0 0x7f0f6c154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f7427b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c155050 con 0x7f0f6c1438f0 2023-06-23T18:36:24.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f7427b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c158510 con 0x7f0f6c0cedf0 2023-06-23T18:36:24.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f7427b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c15ba80 con 0x7f0f6c0d08c0 2023-06-23T18:36:24.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f72818700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f6c1438f0 0x7f0f6c154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f72818700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f6c1438f0 0x7f0f6c154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40242/0 (socket says 172.21.15.79:40242) 2023-06-23T18:36:24.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f72818700 1 -- 172.21.15.79:0/4144580024 learned_addr learned my addr 172.21.15.79:0/4144580024 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:24.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f71816700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f6c0d08c0 0x7f0f6c09a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f72017700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f6c0cedf0 0x7f0f6c15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f72818700 1 -- 172.21.15.79:0/4144580024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f6c0d08c0 msgr2=0x7f0f6c09a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f72818700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f6c0d08c0 0x7f0f6c09a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f72818700 1 -- 172.21.15.79:0/4144580024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f6c0cedf0 msgr2=0x7f0f6c15b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f72818700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f6c0cedf0 0x7f0f6c15b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f72818700 1 -- 172.21.15.79:0/4144580024 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f68009550 con 0x7f0f6c1438f0 2023-06-23T18:36:24.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f71816700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f6c0d08c0 0x7f0f6c09a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:24.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f72818700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f6c1438f0 0x7f0f6c154b10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0f64007e60 tx=0x7f0f6400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:24.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f62ffd700 1 -- 172.21.15.79:0/4144580024 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6400ede0 con 0x7f0f6c1438f0 2023-06-23T18:36:24.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f62ffd700 1 -- 172.21.15.79:0/4144580024 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f64014420 con 0x7f0f6c1438f0 2023-06-23T18:36:24.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f62ffd700 1 -- 172.21.15.79:0/4144580024 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f640147d0 con 0x7f0f6c1438f0 2023-06-23T18:36:24.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f7427b700 1 -- 172.21.15.79:0/4144580024 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f6c15bd60 con 0x7f0f6c1438f0 2023-06-23T18:36:24.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.791+0000 7f0f7427b700 1 -- 172.21.15.79:0/4144580024 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6c15c320 con 0x7f0f6c1438f0 2023-06-23T18:36:24.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.792+0000 7f0f7427b700 1 -- 172.21.15.79:0/4144580024 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f6c0672c0 con 0x7f0f6c1438f0 2023-06-23T18:36:24.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.793+0000 7f0f62ffd700 1 -- 172.21.15.79:0/4144580024 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f64014930 con 0x7f0f6c1438f0 2023-06-23T18:36:24.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.793+0000 7f0f62ffd700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0f5006ace0 0x7f0f5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:24.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.793+0000 7f0f62ffd700 1 -- 172.21.15.79:0/4144580024 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0f64089b40 con 0x7f0f6c1438f0 2023-06-23T18:36:24.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.793+0000 7f0f72017700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0f5006ace0 0x7f0f5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:24.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.793+0000 7f0f72017700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0f5006ace0 0x7f0f5006d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c157200 tx=0x7f0f5c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:24.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.797+0000 7f0f62ffd700 1 -- 172.21.15.79:0/4144580024 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0f64054c80 con 0x7f0f6c1438f0 2023-06-23T18:36:24.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.984+0000 7f0f7427b700 1 -- 172.21.15.79:0/4144580024 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f6c0cdb20 con 0x7f0f6c1438f0 2023-06-23T18:36:24.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.985+0000 7f0f62ffd700 1 -- 172.21.15.79:0/4144580024 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f640582a0 con 0x7f0f6c1438f0 2023-06-23T18:36:24.990 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:24.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.987+0000 7f0f7427b700 1 -- 172.21.15.79:0/4144580024 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0f5006ace0 msgr2=0x7f0f5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.987+0000 7f0f7427b700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0f5006ace0 0x7f0f5006d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c157200 tx=0x7f0f5c008040 comp rx=0 tx=0).stop 2023-06-23T18:36:24.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.987+0000 7f0f7427b700 1 -- 172.21.15.79:0/4144580024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f6c1438f0 msgr2=0x7f0f6c154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:24.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.987+0000 7f0f7427b700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f6c1438f0 0x7f0f6c154b10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0f64007e60 tx=0x7f0f6400e3c0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.987+0000 7f0f7427b700 1 -- 172.21.15.79:0/4144580024 shutdown_connections 2023-06-23T18:36:24.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.987+0000 7f0f7427b700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f6c1438f0 0x7f0f6c154b10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.987+0000 7f0f7427b700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0f5006ace0 0x7f0f5006d190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.987+0000 7f0f7427b700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f6c0cedf0 0x7f0f6c15b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.987+0000 7f0f7427b700 1 --2- 172.21.15.79:0/4144580024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f6c0d08c0 0x7f0f6c09a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:24.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.987+0000 7f0f7427b700 1 -- 172.21.15.79:0/4144580024 >> 172.21.15.79:0/4144580024 conn(0x7f0f6c06a7b0 msgr2=0x7f0f6c0b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:24.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.987+0000 7f0f7427b700 1 -- 172.21.15.79:0/4144580024 shutdown_connections 2023-06-23T18:36:24.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:24.987+0000 7f0f7427b700 1 -- 172.21.15.79:0/4144580024 wait complete. 2023-06-23T18:36:25.141 INFO:tasks.cephfs_test_runner:Starting test: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-06-23T18:36:25.143 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values' 2023-06-23T18:36:25.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.259+0000 7f9bd1161700 1 Processor -- start 2023-06-23T18:36:25.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.259+0000 7f9bd1161700 1 -- start start 2023-06-23T18:36:25.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bd1161700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9bcc071490 0x7f9bcc071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:25.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bd1161700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9bcc0aee30 0x7f9bcc071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:25.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bd1161700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bcc1608f0 con 0x7f9bcc0b1270 2023-06-23T18:36:25.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bd1161700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bcc0722e0 con 0x7f9bcc071490 2023-06-23T18:36:25.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bd1161700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bcc160e20 con 0x7f9bcc0aee30 2023-06-23T18:36:25.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bcad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9bcc071490 0x7f9bcc071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:25.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bcad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9bcc071490 0x7f9bcc071860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40246/0 (socket says 172.21.15.79:40246) 2023-06-23T18:36:25.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bcad9d700 1 -- 172.21.15.79:0/886246431 learned_addr learned my addr 172.21.15.79:0/886246431 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:25.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bcb59e700 1 --2- 172.21.15.79:0/886246431 >> v2:172.21.15.155:3300/0 conn(0x7f9bcc0aee30 0x7f9bcc071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:25.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bcad9d700 1 -- 172.21.15.79:0/886246431 >> v2:172.21.15.155:3300/0 conn(0x7f9bcc0aee30 msgr2=0x7f9bcc071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:25.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bcad9d700 1 --2- 172.21.15.79:0/886246431 >> v2:172.21.15.155:3300/0 conn(0x7f9bcc0aee30 0x7f9bcc071da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:25.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bcad9d700 1 -- 172.21.15.79:0/886246431 >> v1:172.21.15.155:6789/0 conn(0x7f9bcc0b1270 legacy=0x7f9bcc069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:25.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bcad9d700 1 -- 172.21.15.79:0/886246431 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc0cd4f0 con 0x7f9bcc071490 2023-06-23T18:36:25.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bcad9d700 1 --2- 172.21.15.79:0/886246431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bcc071490 0x7f9bcc071860 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc071290 tx=0x7f9bc0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da3ed0813ae4e6d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:25.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bc9d9b700 1 -- 172.21.15.79:0/886246431 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0004230 con 0x7f9bcc071490 2023-06-23T18:36:25.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bc9d9b700 1 -- 172.21.15.79:0/886246431 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0004390 con 0x7f9bcc071490 2023-06-23T18:36:25.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.260+0000 7f9bc9d9b700 1 -- 172.21.15.79:0/886246431 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00036a0 con 0x7f9bcc071490 2023-06-23T18:36:25.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.261+0000 7f9bd1161700 1 -- 172.21.15.79:0/886246431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bcc071490 msgr2=0x7f9bcc071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:25.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.261+0000 7f9bd1161700 1 --2- 172.21.15.79:0/886246431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bcc071490 0x7f9bcc071860 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc071290 tx=0x7f9bc0009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:25.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.261+0000 7f9bd1161700 1 -- 172.21.15.79:0/886246431 shutdown_connections 2023-06-23T18:36:25.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.261+0000 7f9bd1161700 1 --2- 172.21.15.79:0/886246431 >> v2:172.21.15.155:3300/0 conn(0x7f9bcc0aee30 0x7f9bcc071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:25.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.261+0000 7f9bd1161700 1 --2- 172.21.15.79:0/886246431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bcc071490 0x7f9bcc071860 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:25.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.261+0000 7f9bd1161700 1 -- 172.21.15.79:0/886246431 >> 172.21.15.79:0/886246431 conn(0x7f9bcc06d660 msgr2=0x7f9bcc06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:25.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.261+0000 7f9bd1161700 1 -- 172.21.15.79:0/886246431 shutdown_connections 2023-06-23T18:36:25.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.261+0000 7f9bd1161700 1 -- 172.21.15.79:0/886246431 wait complete. 2023-06-23T18:36:25.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.262+0000 7f9bd1161700 1 Processor -- start 2023-06-23T18:36:25.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.262+0000 7f9bd1161700 1 -- start start 2023-06-23T18:36:25.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.263+0000 7f9bd1161700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bcc071490 0x7f9bcc1578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:25.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.263+0000 7f9bd1161700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bcc0aee30 0x7f9bcc15d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:25.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.263+0000 7f9bd1161700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bcc0b1270 0x7f9bcc0c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:25.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.263+0000 7f9bd1161700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc1f2980 con 0x7f9bcc071490 2023-06-23T18:36:25.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.263+0000 7f9bd1161700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc15f8b0 con 0x7f9bcc0aee30 2023-06-23T18:36:25.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.263+0000 7f9bd1161700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc1f2b60 con 0x7f9bcc0b1270 2023-06-23T18:36:25.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.263+0000 7f9bcad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bcc071490 0x7f9bcc1578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:25.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.263+0000 7f9bcad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bcc071490 0x7f9bcc1578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40254/0 (socket says 172.21.15.79:40254) 2023-06-23T18:36:25.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.263+0000 7f9bcad9d700 1 -- 172.21.15.79:0/1612150409 learned_addr learned my addr 172.21.15.79:0/1612150409 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:25.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.263+0000 7f9bcb59e700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bcc0b1270 0x7f9bcc0c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:25.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.263+0000 7f9bca59c700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bcc0aee30 0x7f9bcc15d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:25.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.264+0000 7f9bcad9d700 1 -- 172.21.15.79:0/1612150409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bcc0b1270 msgr2=0x7f9bcc0c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:25.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.264+0000 7f9bcad9d700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bcc0b1270 0x7f9bcc0c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:25.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.264+0000 7f9bcad9d700 1 -- 172.21.15.79:0/1612150409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bcc0aee30 msgr2=0x7f9bcc15d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:25.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.264+0000 7f9bcad9d700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bcc0aee30 0x7f9bcc15d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:25.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.264+0000 7f9bcad9d700 1 -- 172.21.15.79:0/1612150409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc0009550 con 0x7f9bcc071490 2023-06-23T18:36:25.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.264+0000 7f9bca59c700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bcc0aee30 0x7f9bcc15d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:25.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.264+0000 7f9bcad9d700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bcc071490 0x7f9bcc1578c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc070bc0 tx=0x7f9bc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:25.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.264+0000 7f9bb37fe700 1 -- 172.21.15.79:0/1612150409 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0011630 con 0x7f9bcc071490 2023-06-23T18:36:25.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.264+0000 7f9bb37fe700 1 -- 172.21.15.79:0/1612150409 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0011790 con 0x7f9bcc071490 2023-06-23T18:36:25.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.264+0000 7f9bd1161700 1 -- 172.21.15.79:0/1612150409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc1f2e00 con 0x7f9bcc071490 2023-06-23T18:36:25.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.264+0000 7f9bb37fe700 1 -- 172.21.15.79:0/1612150409 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0011b40 con 0x7f9bcc071490 2023-06-23T18:36:25.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.264+0000 7f9bd1161700 1 -- 172.21.15.79:0/1612150409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc1f33c0 con 0x7f9bcc071490 2023-06-23T18:36:25.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.265+0000 7f9bd1161700 1 -- 172.21.15.79:0/1612150409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc0678d0 con 0x7f9bcc071490 2023-06-23T18:36:25.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.265+0000 7f9bb37fe700 1 -- 172.21.15.79:0/1612150409 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9bc0011ca0 con 0x7f9bcc071490 2023-06-23T18:36:25.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.266+0000 7f9bb37fe700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9bb806ace0 0x7f9bb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:25.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.266+0000 7f9bb37fe700 1 -- 172.21.15.79:0/1612150409 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9bc008aad0 con 0x7f9bcc071490 2023-06-23T18:36:25.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.266+0000 7f9bca59c700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9bb806ace0 0x7f9bb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:25.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.266+0000 7f9bca59c700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9bb806ace0 0x7f9bb806d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc06e780 tx=0x7f9bbc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:25.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.270+0000 7f9bb37fe700 1 -- 172.21.15.79:0/1612150409 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9bc0055b90 con 0x7f9bcc071490 2023-06-23T18:36:25.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:25.530+0000 7f9bd1161700 1 -- 172.21.15.79:0/1612150409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]} v 0) v1 -- 0x7f9bcc0724a0 con 0x7f9bcc071490 2023-06-23T18:36:26.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.021+0000 7f9bb37fe700 1 -- 172.21.15.79:0/1612150409 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]}]=0 v45) v1 ==== 163+0+0 (secure 0 0 0) 0x7f9bc00591b0 con 0x7f9bcc071490 2023-06-23T18:36:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.024+0000 7f9bd1161700 1 -- 172.21.15.79:0/1612150409 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9bb806ace0 msgr2=0x7f9bb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.024+0000 7f9bd1161700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9bb806ace0 0x7f9bb806d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc06e780 tx=0x7f9bbc008040 comp rx=0 tx=0).stop 2023-06-23T18:36:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.024+0000 7f9bd1161700 1 -- 172.21.15.79:0/1612150409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bcc071490 msgr2=0x7f9bcc1578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.024+0000 7f9bd1161700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bcc071490 0x7f9bcc1578c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc070bc0 tx=0x7f9bc0004750 comp rx=0 tx=0).stop 2023-06-23T18:36:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.024+0000 7f9bd1161700 1 -- 172.21.15.79:0/1612150409 shutdown_connections 2023-06-23T18:36:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.024+0000 7f9bd1161700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bcc0b1270 0x7f9bcc0c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.024+0000 7f9bd1161700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bcc071490 0x7f9bcc1578c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.024+0000 7f9bd1161700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9bb806ace0 0x7f9bb806d190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.024+0000 7f9bd1161700 1 --2- 172.21.15.79:0/1612150409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bcc0aee30 0x7f9bcc15d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.024+0000 7f9bd1161700 1 -- 172.21.15.79:0/1612150409 >> 172.21.15.79:0/1612150409 conn(0x7f9bcc06d660 msgr2=0x7f9bcc0cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.024+0000 7f9bd1161700 1 -- 172.21.15.79:0/1612150409 shutdown_connections 2023-06-23T18:36:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.024+0000 7f9bd1161700 1 -- 172.21.15.79:0/1612150409 wait complete. 2023-06-23T18:36:26.042 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-06-23T18:36:26.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.162+0000 7fa228ee7700 1 Processor -- start 2023-06-23T18:36:26.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.162+0000 7fa228ee7700 1 -- start start 2023-06-23T18:36:26.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.163+0000 7fa228ee7700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa2240cc060 0x7fa2240cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.163+0000 7fa228ee7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa224071710 0x7fa224071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.163+0000 7fa228ee7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2240cc9f0 con 0x7fa224156ae0 2023-06-23T18:36:26.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.163+0000 7fa228ee7700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22415a560 con 0x7fa2240cc060 2023-06-23T18:36:26.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.163+0000 7fa228ee7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa224159950 con 0x7fa224071710 2023-06-23T18:36:26.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22259c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa224156ae0 0x7fa22415d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43302/0 (socket says 172.21.15.79:43302) 2023-06-23T18:36:26.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22259c700 1 -- 172.21.15.79:0/2000678711 learned_addr learned my addr 172.21.15.79:0/2000678711 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:26.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa222d9d700 1 --2- 172.21.15.79:0/2000678711 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa224071710 0x7fa224071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:26.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa221d9b700 1 --2- 172.21.15.79:0/2000678711 >> v2:172.21.15.155:3300/0 conn(0x7fa2240cc060 0x7fa2240cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:26.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 -- 172.21.15.79:0/2000678711 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230403837 0 0) 0x7fa218001110 con 0x7fa224156ae0 2023-06-23T18:36:26.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 -- 172.21.15.79:0/2000678711 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa20c0035b0 con 0x7fa224156ae0 2023-06-23T18:36:26.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 -- 172.21.15.79:0/2000678711 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1501368873 0 0) 0x7fa20c0035b0 con 0x7fa224156ae0 2023-06-23T18:36:26.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 -- 172.21.15.79:0/2000678711 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa224071710 msgr2=0x7fa224071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:26.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 --2- 172.21.15.79:0/2000678711 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa224071710 0x7fa224071b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 -- 172.21.15.79:0/2000678711 >> v2:172.21.15.155:3300/0 conn(0x7fa2240cc060 msgr2=0x7fa2240cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:26.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 --2- 172.21.15.79:0/2000678711 >> v2:172.21.15.155:3300/0 conn(0x7fa2240cc060 0x7fa2240cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 -- 172.21.15.79:0/2000678711 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa224159c30 con 0x7fa224156ae0 2023-06-23T18:36:26.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 -- 172.21.15.79:0/2000678711 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fa218001800 con 0x7fa224156ae0 2023-06-23T18:36:26.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 -- 172.21.15.79:0/2000678711 >> v1:172.21.15.79:6789/0 conn(0x7fa224156ae0 legacy=0x7fa22415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:26.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 --2- 172.21.15.79:0/2000678711 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa224071710 0x7fa20c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 --2- 172.21.15.79:0/2000678711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa20c003f80 0x7fa20c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 --2- 172.21.15.79:0/2000678711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa20c004d90 0x7fa20c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 -- 172.21.15.79:0/2000678711 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22415a560 con 0x7fa20c003f80 2023-06-23T18:36:26.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 -- 172.21.15.79:0/2000678711 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa224159950 con 0x7fa20c004d90 2023-06-23T18:36:26.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22159a700 1 -- 172.21.15.79:0/2000678711 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c00ba10 con 0x7fa224071710 2023-06-23T18:36:26.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa221d9b700 1 --2- 172.21.15.79:0/2000678711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa20c003f80 0x7fa20c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:26.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa22259c700 1 --2- 172.21.15.79:0/2000678711 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa224071710 0x7fa20c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:26.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.164+0000 7fa222d9d700 1 --2- 172.21.15.79:0/2000678711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa20c004d90 0x7fa20c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:26.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.165+0000 7fa221d9b700 1 -- 172.21.15.79:0/2000678711 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa224071710 msgr2=0x7fa20c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:26.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.165+0000 7fa221d9b700 1 --2- 172.21.15.79:0/2000678711 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa224071710 0x7fa20c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.165+0000 7fa221d9b700 1 -- 172.21.15.79:0/2000678711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa20c004d90 msgr2=0x7fa20c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:26.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.165+0000 7fa221d9b700 1 --2- 172.21.15.79:0/2000678711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa20c004d90 0x7fa20c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.165+0000 7fa221d9b700 1 -- 172.21.15.79:0/2000678711 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c0035b0 con 0x7fa20c003f80 2023-06-23T18:36:26.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.165+0000 7fa221d9b700 1 --2- 172.21.15.79:0/2000678711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa20c003f80 0x7fa20c004850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa214009b60 tx=0x7fa20c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe7b7c78c0949da server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:26.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.165+0000 7fa22159a700 1 -- 172.21.15.79:0/2000678711 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400e6c0 con 0x7fa20c003f80 2023-06-23T18:36:26.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.165+0000 7fa22159a700 1 -- 172.21.15.79:0/2000678711 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21400e820 con 0x7fa20c003f80 2023-06-23T18:36:26.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.165+0000 7fa22159a700 1 -- 172.21.15.79:0/2000678711 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400eb90 con 0x7fa20c003f80 2023-06-23T18:36:26.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.165+0000 7fa228ee7700 1 -- 172.21.15.79:0/2000678711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa20c003f80 msgr2=0x7fa20c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:26.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.165+0000 7fa228ee7700 1 --2- 172.21.15.79:0/2000678711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa20c003f80 0x7fa20c004850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa214009b60 tx=0x7fa20c005880 comp rx=0 tx=0).stop 2023-06-23T18:36:26.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.165+0000 7fa222d9d700 1 -- 172.21.15.79:0/2000678711 reap_dead start 2023-06-23T18:36:26.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.166+0000 7fa228ee7700 1 -- 172.21.15.79:0/2000678711 shutdown_connections 2023-06-23T18:36:26.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.166+0000 7fa228ee7700 1 -- 172.21.15.79:0/2000678711 >> 172.21.15.79:0/2000678711 conn(0x7fa22406afa0 msgr2=0x7fa22406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:26.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.166+0000 7fa228ee7700 1 -- 172.21.15.79:0/2000678711 shutdown_connections 2023-06-23T18:36:26.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.166+0000 7fa228ee7700 1 -- 172.21.15.79:0/2000678711 wait complete. 2023-06-23T18:36:26.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.167+0000 7fa228ee7700 1 Processor -- start 2023-06-23T18:36:26.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.167+0000 7fa228ee7700 1 -- start start 2023-06-23T18:36:26.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa228ee7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa20c003f80 0x7fa2240c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa228ee7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2240b26a0 0x7fa2240c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa228ee7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2240afb20 0x7fa2240b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa228ee7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2240c75b0 con 0x7fa2240b26a0 2023-06-23T18:36:26.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa228ee7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2240c61c0 con 0x7fa2240afb20 2023-06-23T18:36:26.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa228ee7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2240b2c10 con 0x7fa20c003f80 2023-06-23T18:36:26.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa221d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2240b26a0 0x7fa2240c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:26.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa221d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2240b26a0 0x7fa2240c5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40274/0 (socket says 172.21.15.79:40274) 2023-06-23T18:36:26.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa221d9b700 1 -- 172.21.15.79:0/978016157 learned_addr learned my addr 172.21.15.79:0/978016157 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:26.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa221d9b700 1 -- 172.21.15.79:0/978016157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa20c003f80 msgr2=0x7fa2240c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:26.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa221d9b700 1 --2- 172.21.15.79:0/978016157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa20c003f80 0x7fa2240c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa221d9b700 1 -- 172.21.15.79:0/978016157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2240afb20 msgr2=0x7fa2240b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:26.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa222d9d700 1 --2- 172.21.15.79:0/978016157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2240afb20 0x7fa2240b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:26.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa221d9b700 1 --2- 172.21.15.79:0/978016157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2240afb20 0x7fa2240b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa221d9b700 1 -- 172.21.15.79:0/978016157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2240b2ef0 con 0x7fa2240b26a0 2023-06-23T18:36:26.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa221d9b700 1 --2- 172.21.15.79:0/978016157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2240b26a0 0x7fa2240c5c80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa2140090b0 tx=0x7fa2140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:26.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa212ffd700 1 -- 172.21.15.79:0/978016157 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400e6c0 con 0x7fa2240b26a0 2023-06-23T18:36:26.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa212ffd700 1 -- 172.21.15.79:0/978016157 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa214003ad0 con 0x7fa2240b26a0 2023-06-23T18:36:26.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa212ffd700 1 -- 172.21.15.79:0/978016157 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214003e80 con 0x7fa2240b26a0 2023-06-23T18:36:26.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa228ee7700 1 -- 172.21.15.79:0/978016157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2240b31d0 con 0x7fa2240b26a0 2023-06-23T18:36:26.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.168+0000 7fa228ee7700 1 -- 172.21.15.79:0/978016157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2240b3790 con 0x7fa2240b26a0 2023-06-23T18:36:26.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.169+0000 7fa228ee7700 1 -- 172.21.15.79:0/978016157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2240b3920 con 0x7fa2240b26a0 2023-06-23T18:36:26.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.170+0000 7fa212ffd700 1 -- 172.21.15.79:0/978016157 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa214013470 con 0x7fa2240b26a0 2023-06-23T18:36:26.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.170+0000 7fa212ffd700 1 --2- 172.21.15.79:0/978016157 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa20406ace0 0x7fa20406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.170+0000 7fa212ffd700 1 -- 172.21.15.79:0/978016157 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa21408b120 con 0x7fa2240b26a0 2023-06-23T18:36:26.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.170+0000 7fa22259c700 1 --2- 172.21.15.79:0/978016157 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa20406ace0 0x7fa20406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:26.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.171+0000 7fa22259c700 1 --2- 172.21.15.79:0/978016157 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa20406ace0 0x7fa20406d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa2180012b0 tx=0x7fa218001d00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:26.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.174+0000 7fa212ffd700 1 -- 172.21.15.79:0/978016157 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa2140562e0 con 0x7fa2240b26a0 2023-06-23T18:36:26.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.362+0000 7fa228ee7700 1 -- 172.21.15.79:0/978016157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa2240672c0 con 0x7fa2240b26a0 2023-06-23T18:36:26.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.368+0000 7fa212ffd700 1 -- 172.21.15.79:0/978016157 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa214059900 con 0x7fa2240b26a0 2023-06-23T18:36:26.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.370+0000 7fa228ee7700 1 -- 172.21.15.79:0/978016157 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa20406ace0 msgr2=0x7fa20406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:26.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.370+0000 7fa228ee7700 1 --2- 172.21.15.79:0/978016157 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa20406ace0 0x7fa20406d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa2180012b0 tx=0x7fa218001d00 comp rx=0 tx=0).stop 2023-06-23T18:36:26.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.370+0000 7fa228ee7700 1 -- 172.21.15.79:0/978016157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2240b26a0 msgr2=0x7fa2240c5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:26.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.370+0000 7fa228ee7700 1 --2- 172.21.15.79:0/978016157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2240b26a0 0x7fa2240c5c80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa2140090b0 tx=0x7fa2140047c0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.371+0000 7fa228ee7700 1 -- 172.21.15.79:0/978016157 shutdown_connections 2023-06-23T18:36:26.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.371+0000 7fa228ee7700 1 --2- 172.21.15.79:0/978016157 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa20406ace0 0x7fa20406d190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.371+0000 7fa228ee7700 1 --2- 172.21.15.79:0/978016157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2240afb20 0x7fa2240b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.371+0000 7fa228ee7700 1 --2- 172.21.15.79:0/978016157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa20c003f80 0x7fa2240c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.371+0000 7fa228ee7700 1 --2- 172.21.15.79:0/978016157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2240b26a0 0x7fa2240c5c80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.371+0000 7fa228ee7700 1 -- 172.21.15.79:0/978016157 >> 172.21.15.79:0/978016157 conn(0x7fa22406afa0 msgr2=0x7fa2241585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:26.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.371+0000 7fa228ee7700 1 -- 172.21.15.79:0/978016157 shutdown_connections 2023-06-23T18:36:26.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.371+0000 7fa228ee7700 1 -- 172.21.15.79:0/978016157 wait complete. 2023-06-23T18:36:26.387 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-06-23T18:36:26.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-06-23T18:36:26.429 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-06-23T18:36:26.478 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-06-23T18:36:26.478 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:36:26.479 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:36:26.496 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:36:26.496 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:36:26.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:26.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.708+0000 7f1384496700 1 Processor -- start 2023-06-23T18:36:26.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.708+0000 7f1384496700 1 -- start start 2023-06-23T18:36:26.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.709+0000 7f1384496700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f137c15a380 0x7f137c0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.709+0000 7f1384496700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f137c0cea30 0x7f137c0b1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.709+0000 7f1384496700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f137c0cee70 con 0x7f137c15a750 2023-06-23T18:36:26.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.709+0000 7f1384496700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137c0b1d40 con 0x7f137c0cea30 2023-06-23T18:36:26.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.709+0000 7f1384496700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0b2020 con 0x7f137c15a380 2023-06-23T18:36:26.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.709+0000 7f1382232700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f137c15a380 0x7f137c0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:26.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.709+0000 7f1382a33700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f137c0cea30 0x7f137c0b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:26.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.709+0000 7f1382232700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f137c15a380 0x7f137c0ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:33050/0 (socket says 172.21.15.79:33050) 2023-06-23T18:36:26.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.709+0000 7f1382232700 1 -- 172.21.15.79:0/3966539091 learned_addr learned my addr 172.21.15.79:0/3966539091 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:26.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.709+0000 7f1382a33700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f137c0cea30 0x7f137c0b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:46506/0 (socket says 172.21.15.79:46506) 2023-06-23T18:36:26.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.709+0000 7f1381a31700 1 --1- 172.21.15.79:0/3966539091 >> v1:172.21.15.155:6789/0 conn(0x7f137c15a750 0x7f137c0ade50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:51590/0 (socket says 172.21.15.79:51590) 2023-06-23T18:36:26.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.710+0000 7f1382a33700 1 -- 172.21.15.79:0/3966539091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f137c15a380 msgr2=0x7f137c0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:26.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.710+0000 7f1382a33700 1 --2- 172.21.15.79:0/3966539091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f137c15a380 0x7f137c0ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.710+0000 7f1382a33700 1 -- 172.21.15.79:0/3966539091 >> v1:172.21.15.155:6789/0 conn(0x7f137c15a750 legacy=0x7f137c0ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:26.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.710+0000 7f1382a33700 1 -- 172.21.15.79:0/3966539091 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c0ad650 con 0x7f137c0cea30 2023-06-23T18:36:26.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.710+0000 7f1382232700 1 --2- 172.21.15.79:0/3966539091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f137c15a380 0x7f137c0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:26.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.710+0000 7f1382a33700 1 --2- 172.21.15.79:0/3966539091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f137c0cea30 0x7f137c0b1010 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1374009d90 tx=0x7f137400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5b435d71da74c4f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:26.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.710+0000 7f1381230700 1 -- 172.21.15.79:0/3966539091 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137400e070 con 0x7f137c0cea30 2023-06-23T18:36:26.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.710+0000 7f1381230700 1 -- 172.21.15.79:0/3966539091 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137400bb20 con 0x7f137c0cea30 2023-06-23T18:36:26.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.710+0000 7f1381230700 1 -- 172.21.15.79:0/3966539091 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374012430 con 0x7f137c0cea30 2023-06-23T18:36:26.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.711+0000 7f1384496700 1 -- 172.21.15.79:0/3966539091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f137c0cea30 msgr2=0x7f137c0b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:26.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.711+0000 7f1384496700 1 --2- 172.21.15.79:0/3966539091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f137c0cea30 0x7f137c0b1010 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1374009d90 tx=0x7f137400b330 comp rx=0 tx=0).stop 2023-06-23T18:36:26.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.711+0000 7f1384496700 1 -- 172.21.15.79:0/3966539091 shutdown_connections 2023-06-23T18:36:26.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.711+0000 7f1384496700 1 --2- 172.21.15.79:0/3966539091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f137c0cea30 0x7f137c0b1010 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.711+0000 7f1384496700 1 --2- 172.21.15.79:0/3966539091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f137c15a380 0x7f137c0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.711+0000 7f1384496700 1 -- 172.21.15.79:0/3966539091 >> 172.21.15.79:0/3966539091 conn(0x7f137c089fc0 msgr2=0x7f137c099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:26.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.711+0000 7f1384496700 1 -- 172.21.15.79:0/3966539091 shutdown_connections 2023-06-23T18:36:26.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.711+0000 7f1384496700 1 -- 172.21.15.79:0/3966539091 wait complete. 2023-06-23T18:36:26.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.712+0000 7f1384496700 1 Processor -- start 2023-06-23T18:36:26.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.712+0000 7f1384496700 1 -- start start 2023-06-23T18:36:26.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1384496700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f137c0cea30 0x7f137c0c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1384496700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c15a380 0x7f137c1f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1384496700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f137c15a750 0x7f137c15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1384496700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0c7350 con 0x7f137c15a380 2023-06-23T18:36:26.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1384496700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0c6ec0 con 0x7f137c0cea30 2023-06-23T18:36:26.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1384496700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c1f3e90 con 0x7f137c15a750 2023-06-23T18:36:26.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1381a31700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c15a380 0x7f137c1f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:26.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1381a31700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c15a380 0x7f137c1f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40286/0 (socket says 172.21.15.79:40286) 2023-06-23T18:36:26.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1381a31700 1 -- 172.21.15.79:0/1641888833 learned_addr learned my addr 172.21.15.79:0/1641888833 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:26.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1382a33700 1 --2- 172.21.15.79:0/1641888833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f137c15a750 0x7f137c15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:26.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1382232700 1 --2- 172.21.15.79:0/1641888833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f137c0cea30 0x7f137c0c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:26.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1381a31700 1 -- 172.21.15.79:0/1641888833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f137c15a750 msgr2=0x7f137c15cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:26.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1381a31700 1 --2- 172.21.15.79:0/1641888833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f137c15a750 0x7f137c15cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1381a31700 1 -- 172.21.15.79:0/1641888833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f137c0cea30 msgr2=0x7f137c0c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:26.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1381a31700 1 --2- 172.21.15.79:0/1641888833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f137c0cea30 0x7f137c0c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.713+0000 7f1381a31700 1 -- 172.21.15.79:0/1641888833 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c009500 con 0x7f137c15a380 2023-06-23T18:36:26.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.714+0000 7f1381a31700 1 --2- 172.21.15.79:0/1641888833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c15a380 0x7f137c1f3950 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f137800c410 tx=0x7f137800c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:26.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.714+0000 7f1372ffd700 1 -- 172.21.15.79:0/1641888833 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378005cb0 con 0x7f137c15a380 2023-06-23T18:36:26.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.714+0000 7f1372ffd700 1 -- 172.21.15.79:0/1641888833 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378005e10 con 0x7f137c15a380 2023-06-23T18:36:26.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.714+0000 7f1384496700 1 -- 172.21.15.79:0/1641888833 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1374009a70 con 0x7f137c15a380 2023-06-23T18:36:26.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.714+0000 7f1372ffd700 1 -- 172.21.15.79:0/1641888833 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13780072f0 con 0x7f137c15a380 2023-06-23T18:36:26.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.714+0000 7f1384496700 1 -- 172.21.15.79:0/1641888833 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137c0c6590 con 0x7f137c15a380 2023-06-23T18:36:26.725 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.715+0000 7f1384496700 1 -- 172.21.15.79:0/1641888833 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f137c0c7030 con 0x7f137c15a380 2023-06-23T18:36:26.725 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.715+0000 7f1372ffd700 1 -- 172.21.15.79:0/1641888833 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1378007450 con 0x7f137c15a380 2023-06-23T18:36:26.725 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.715+0000 7f1372ffd700 1 --2- 172.21.15.79:0/1641888833 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f136006ace0 0x7f136006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:26.725 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.716+0000 7f1382232700 1 --2- 172.21.15.79:0/1641888833 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f136006ace0 0x7f136006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:26.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.716+0000 7f1372ffd700 1 -- 172.21.15.79:0/1641888833 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1378088760 con 0x7f137c15a380 2023-06-23T18:36:26.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.716+0000 7f1382232700 1 --2- 172.21.15.79:0/1641888833 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f136006ace0 0x7f136006d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f136c006fd0 tx=0x7f136c009290 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:26.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.720+0000 7f1372ffd700 1 -- 172.21.15.79:0/1641888833 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1378053810 con 0x7f137c15a380 2023-06-23T18:36:26.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.910+0000 7f1384496700 1 -- 172.21.15.79:0/1641888833 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f137c04e240 con 0x7f137c15a380 2023-06-23T18:36:26.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.911+0000 7f1372ffd700 1 -- 172.21.15.79:0/1641888833 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7f1378056e30 con 0x7f137c15a380 2023-06-23T18:36:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:26.916 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:26.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.913+0000 7f1384496700 1 -- 172.21.15.79:0/1641888833 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f136006ace0 msgr2=0x7f136006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:26.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.913+0000 7f1384496700 1 --2- 172.21.15.79:0/1641888833 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f136006ace0 0x7f136006d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f136c006fd0 tx=0x7f136c009290 comp rx=0 tx=0).stop 2023-06-23T18:36:26.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.913+0000 7f1384496700 1 -- 172.21.15.79:0/1641888833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c15a380 msgr2=0x7f137c1f3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:26.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.913+0000 7f1384496700 1 --2- 172.21.15.79:0/1641888833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c15a380 0x7f137c1f3950 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f137800c410 tx=0x7f137800c720 comp rx=0 tx=0).stop 2023-06-23T18:36:26.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.913+0000 7f1384496700 1 -- 172.21.15.79:0/1641888833 shutdown_connections 2023-06-23T18:36:26.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.913+0000 7f1384496700 1 --2- 172.21.15.79:0/1641888833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f137c15a750 0x7f137c15cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.913+0000 7f1384496700 1 --2- 172.21.15.79:0/1641888833 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f136006ace0 0x7f136006d190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.913+0000 7f1384496700 1 --2- 172.21.15.79:0/1641888833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f137c0cea30 0x7f137c0c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.913+0000 7f1384496700 1 --2- 172.21.15.79:0/1641888833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c15a380 0x7f137c1f3950 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:26.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.913+0000 7f1384496700 1 -- 172.21.15.79:0/1641888833 >> 172.21.15.79:0/1641888833 conn(0x7f137c089fc0 msgr2=0x7f137c0afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:26.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.913+0000 7f1384496700 1 -- 172.21.15.79:0/1641888833 shutdown_connections 2023-06-23T18:36:26.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:26.913+0000 7f1384496700 1 -- 172.21.15.79:0/1641888833 wait complete. 2023-06-23T18:36:26.933 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:26.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:27.059 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.054+0000 7f9a08bb0700 1 Processor -- start 2023-06-23T18:36:27.060 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.054+0000 7f9a08bb0700 1 -- start start 2023-06-23T18:36:27.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.055+0000 7f9a08bb0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a0406a5a0 0x7f9a041608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.055+0000 7f9a08bb0700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a040cdc90 con 0x7f9a041569c0 2023-06-23T18:36:27.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.055+0000 7f9a08bb0700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0415d0f0 con 0x7f9a040cd440 2023-06-23T18:36:27.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.055+0000 7f9a08bb0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040cf570 con 0x7f9a0406a5a0 2023-06-23T18:36:27.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.055+0000 7f9a0259c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9a041569c0 0x7f9a04069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43334/0 (socket says 172.21.15.79:43334) 2023-06-23T18:36:27.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.055+0000 7f9a0259c700 1 -- 172.21.15.79:0/1730064931 learned_addr learned my addr 172.21.15.79:0/1730064931 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:27.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.055+0000 7f9a0159a700 1 -- 172.21.15.79:0/1730064931 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468163562 0 0) 0x7f99f8001110 con 0x7f9a041569c0 2023-06-23T18:36:27.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.055+0000 7f9a0159a700 1 -- 172.21.15.79:0/1730064931 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99ec0035b0 con 0x7f9a041569c0 2023-06-23T18:36:27.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.055+0000 7f9a01d9b700 1 --2- 172.21.15.79:0/1730064931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a0406a5a0 0x7f9a041608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 -- 172.21.15.79:0/1730064931 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2960044888 0 0) 0x7f99ec0035b0 con 0x7f9a041569c0 2023-06-23T18:36:27.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 -- 172.21.15.79:0/1730064931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a0406a5a0 msgr2=0x7f9a041608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:27.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 --2- 172.21.15.79:0/1730064931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a0406a5a0 0x7f9a041608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 -- 172.21.15.79:0/1730064931 >> v1:172.21.15.155:6789/0 conn(0x7f9a040cd440 legacy=0x7f9a040b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 -- 172.21.15.79:0/1730064931 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04156d90 con 0x7f9a041569c0 2023-06-23T18:36:27.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 -- 172.21.15.79:0/1730064931 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f99f8001800 con 0x7f9a041569c0 2023-06-23T18:36:27.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 -- 172.21.15.79:0/1730064931 >> v1:172.21.15.79:6789/0 conn(0x7f9a041569c0 legacy=0x7f9a04069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 --2- 172.21.15.79:0/1730064931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a0406a5a0 0x7f99ec003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 --2- 172.21.15.79:0/1730064931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99ec003fb0 0x7f99ec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 --2- 172.21.15.79:0/1730064931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99ec004d90 0x7f99ec005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 -- 172.21.15.79:0/1730064931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec00bac0 con 0x7f99ec004d90 2023-06-23T18:36:27.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 -- 172.21.15.79:0/1730064931 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec00bc30 con 0x7f99ec003fb0 2023-06-23T18:36:27.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 -- 172.21.15.79:0/1730064931 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec00bdf0 con 0x7f9a0406a5a0 2023-06-23T18:36:27.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a02d9d700 1 --2- 172.21.15.79:0/1730064931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99ec004d90 0x7f99ec005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0259c700 1 --2- 172.21.15.79:0/1730064931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a0406a5a0 0x7f99ec003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a01d9b700 1 --2- 172.21.15.79:0/1730064931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99ec003fb0 0x7f99ec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a02d9d700 1 -- 172.21.15.79:0/1730064931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a0406a5a0 msgr2=0x7f99ec003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:27.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a02d9d700 1 --2- 172.21.15.79:0/1730064931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a0406a5a0 0x7f99ec003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a02d9d700 1 -- 172.21.15.79:0/1730064931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99ec003fb0 msgr2=0x7f99ec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:27.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a02d9d700 1 --2- 172.21.15.79:0/1730064931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99ec003fb0 0x7f99ec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a02d9d700 1 -- 172.21.15.79:0/1730064931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec0035b0 con 0x7f99ec004d90 2023-06-23T18:36:27.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a02d9d700 1 --2- 172.21.15.79:0/1730064931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99ec004d90 0x7f99ec005200 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9a040cd8e0 tx=0x7f99ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ed450962ec3380 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:27.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 -- 172.21.15.79:0/1730064931 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc001db0 con 0x7f99ec004d90 2023-06-23T18:36:27.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 -- 172.21.15.79:0/1730064931 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc00cc70 con 0x7f99ec004d90 2023-06-23T18:36:27.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.056+0000 7f9a0159a700 1 -- 172.21.15.79:0/1730064931 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc003a00 con 0x7f99ec004d90 2023-06-23T18:36:27.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.057+0000 7f9a08bb0700 1 -- 172.21.15.79:0/1730064931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99ec004d90 msgr2=0x7f99ec005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.057+0000 7f9a08bb0700 1 --2- 172.21.15.79:0/1730064931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99ec004d90 0x7f99ec005200 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9a040cd8e0 tx=0x7f99ec005880 comp rx=0 tx=0).stop 2023-06-23T18:36:27.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.057+0000 7f9a02d9d700 1 -- 172.21.15.79:0/1730064931 reap_dead start 2023-06-23T18:36:27.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.057+0000 7f9a08bb0700 1 -- 172.21.15.79:0/1730064931 shutdown_connections 2023-06-23T18:36:27.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.057+0000 7f9a08bb0700 1 -- 172.21.15.79:0/1730064931 >> 172.21.15.79:0/1730064931 conn(0x7f9a0406afa0 msgr2=0x7f9a0406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:27.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.057+0000 7f9a08bb0700 1 -- 172.21.15.79:0/1730064931 shutdown_connections 2023-06-23T18:36:27.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.057+0000 7f9a08bb0700 1 -- 172.21.15.79:0/1730064931 wait complete. 2023-06-23T18:36:27.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.058+0000 7f9a08bb0700 1 Processor -- start 2023-06-23T18:36:27.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.059+0000 7f9a08bb0700 1 -- start start 2023-06-23T18:36:27.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.059+0000 7f9a08bb0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99ec004d90 0x7f9a040ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.059+0000 7f9a08bb0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a041f5e60 0x7f9a04148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.059+0000 7f9a08bb0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a040ad530 0x7f9a040ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.059+0000 7f9a08bb0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040adf10 con 0x7f99ec004d90 2023-06-23T18:36:27.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.059+0000 7f9a08bb0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a04149150 con 0x7f9a040ad530 2023-06-23T18:36:27.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.059+0000 7f9a08bb0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040abf40 con 0x7f9a041f5e60 2023-06-23T18:36:27.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.059+0000 7f9a0259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99ec004d90 0x7f9a040ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.059+0000 7f9a0259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99ec004d90 0x7f9a040ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40304/0 (socket says 172.21.15.79:40304) 2023-06-23T18:36:27.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.059+0000 7f9a0259c700 1 -- 172.21.15.79:0/3513158142 learned_addr learned my addr 172.21.15.79:0/3513158142 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:27.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f9a02d9d700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a040ad530 0x7f9a040ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f9a01d9b700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a041f5e60 0x7f9a04148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f9a0259c700 1 -- 172.21.15.79:0/3513158142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a041f5e60 msgr2=0x7f9a04148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f9a0259c700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a041f5e60 0x7f9a04148c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f9a0259c700 1 -- 172.21.15.79:0/3513158142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a040ad530 msgr2=0x7f9a040ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f9a0259c700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a040ad530 0x7f9a040ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f9a0259c700 1 -- 172.21.15.79:0/3513158142 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a040ac220 con 0x7f99ec004d90 2023-06-23T18:36:27.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f9a01d9b700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a041f5e60 0x7f9a04148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:27.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f9a02d9d700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a040ad530 0x7f9a040ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:27.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f9a0259c700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99ec004d90 0x7f9a040ab8b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f99f80072f0 tx=0x7f99f80076b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:27.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f99f2ffd700 1 -- 172.21.15.79:0/3513158142 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f800f870 con 0x7f99ec004d90 2023-06-23T18:36:27.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f99f2ffd700 1 -- 172.21.15.79:0/3513158142 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f800fa10 con 0x7f99ec004d90 2023-06-23T18:36:27.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f9a08bb0700 1 -- 172.21.15.79:0/3513158142 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a040ac500 con 0x7f99ec004d90 2023-06-23T18:36:27.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f99f2ffd700 1 -- 172.21.15.79:0/3513158142 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f800fdc0 con 0x7f99ec004d90 2023-06-23T18:36:27.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.060+0000 7f9a08bb0700 1 -- 172.21.15.79:0/3513158142 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a040acac0 con 0x7f99ec004d90 2023-06-23T18:36:27.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.061+0000 7f9a08bb0700 1 -- 172.21.15.79:0/3513158142 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a040b26a0 con 0x7f99ec004d90 2023-06-23T18:36:27.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.061+0000 7f99f2ffd700 1 -- 172.21.15.79:0/3513158142 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f99f801d460 con 0x7f99ec004d90 2023-06-23T18:36:27.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.062+0000 7f99f2ffd700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f99e406ace0 0x7f99e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.062+0000 7f99f2ffd700 1 -- 172.21.15.79:0/3513158142 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f99f80894a0 con 0x7f99ec004d90 2023-06-23T18:36:27.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.062+0000 7f9a01d9b700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f99e406ace0 0x7f99e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.062+0000 7f9a01d9b700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f99e406ace0 0x7f99e406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9a04155d40 tx=0x7f99f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:27.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.066+0000 7f99f2ffd700 1 -- 172.21.15.79:0/3513158142 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9a040b26a0 con 0x7f99ec004d90 2023-06-23T18:36:27.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.251+0000 7f9a08bb0700 1 -- 172.21.15.79:0/3513158142 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9a040cdc90 con 0x7f99ec004d90 2023-06-23T18:36:27.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.251+0000 7f99f2ffd700 1 -- 172.21.15.79:0/3513158142 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9a040cdc90 con 0x7f99ec004d90 2023-06-23T18:36:27.256 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:27.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.253+0000 7f9a08bb0700 1 -- 172.21.15.79:0/3513158142 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f99e406ace0 msgr2=0x7f99e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.253+0000 7f9a08bb0700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f99e406ace0 0x7f99e406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9a04155d40 tx=0x7f99f4008040 comp rx=0 tx=0).stop 2023-06-23T18:36:27.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.253+0000 7f9a08bb0700 1 -- 172.21.15.79:0/3513158142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99ec004d90 msgr2=0x7f9a040ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.253+0000 7f9a08bb0700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99ec004d90 0x7f9a040ab8b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f99f80072f0 tx=0x7f99f80076b0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.253+0000 7f9a08bb0700 1 -- 172.21.15.79:0/3513158142 shutdown_connections 2023-06-23T18:36:27.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.253+0000 7f9a08bb0700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f99e406ace0 0x7f99e406d190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.253+0000 7f9a08bb0700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a040ad530 0x7f9a040ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.253+0000 7f9a08bb0700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99ec004d90 0x7f9a040ab8b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.253+0000 7f9a08bb0700 1 --2- 172.21.15.79:0/3513158142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a041f5e60 0x7f9a04148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.253+0000 7f9a08bb0700 1 -- 172.21.15.79:0/3513158142 >> 172.21.15.79:0/3513158142 conn(0x7f9a0406afa0 msgr2=0x7f9a0406dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:27.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.253+0000 7f9a08bb0700 1 -- 172.21.15.79:0/3513158142 shutdown_connections 2023-06-23T18:36:27.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.253+0000 7f9a08bb0700 1 -- 172.21.15.79:0/3513158142 wait complete. 2023-06-23T18:36:27.270 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:27.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.388+0000 7f97ce89b700 1 Processor -- start 2023-06-23T18:36:27.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.388+0000 7f97ce89b700 1 -- start start 2023-06-23T18:36:27.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97ce89b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f97c815c2a0 0x7f97c80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97ce89b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f97c806a5a0 0x7f97c80c9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97ce89b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c815ffe0 con 0x7f97c80b1410 2023-06-23T18:36:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97ce89b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c81602c0 con 0x7f97c815c2a0 2023-06-23T18:36:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97ce89b700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c81605a0 con 0x7f97c806a5a0 2023-06-23T18:36:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97c7fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f97c815c2a0 0x7f97c80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97c77fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f97c80b1410 0x7f97c8069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43354/0 (socket says 172.21.15.79:43354) 2023-06-23T18:36:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97c7fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f97c815c2a0 0x7f97c80ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40312/0 (socket says 172.21.15.79:40312) 2023-06-23T18:36:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97c77fe700 1 -- 172.21.15.79:0/1385165532 learned_addr learned my addr 172.21.15.79:0/1385165532 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97cce38700 1 --2- 172.21.15.79:0/1385165532 >> v2:172.21.15.155:3300/0 conn(0x7f97c806a5a0 0x7f97c80c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97c7fff700 1 -- 172.21.15.79:0/1385165532 >> v2:172.21.15.155:3300/0 conn(0x7f97c806a5a0 msgr2=0x7f97c80c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97c7fff700 1 --2- 172.21.15.79:0/1385165532 >> v2:172.21.15.155:3300/0 conn(0x7f97c806a5a0 0x7f97c80c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97c7fff700 1 -- 172.21.15.79:0/1385165532 >> v1:172.21.15.79:6789/0 conn(0x7f97c80b1410 legacy=0x7f97c8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97c6ffd700 1 -- 172.21.15.79:0/1385165532 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941018463 0 0) 0x7f97c815ffe0 con 0x7f97c80b1410 2023-06-23T18:36:27.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97c7fff700 1 -- 172.21.15.79:0/1385165532 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c8160880 con 0x7f97c815c2a0 2023-06-23T18:36:27.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.389+0000 7f97c7fff700 1 --2- 172.21.15.79:0/1385165532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97c815c2a0 0x7f97c80ce930 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009840 tx=0x7f97bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff906170c1be6df server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:27.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.390+0000 7f97c6ffd700 1 -- 172.21.15.79:0/1385165532 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc011830 con 0x7f97c815c2a0 2023-06-23T18:36:27.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.390+0000 7f97c6ffd700 1 -- 172.21.15.79:0/1385165532 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc011990 con 0x7f97c815c2a0 2023-06-23T18:36:27.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.390+0000 7f97c6ffd700 1 -- 172.21.15.79:0/1385165532 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc011d20 con 0x7f97c815c2a0 2023-06-23T18:36:27.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.390+0000 7f97ce89b700 1 -- 172.21.15.79:0/1385165532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97c815c2a0 msgr2=0x7f97c80ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.390+0000 7f97ce89b700 1 --2- 172.21.15.79:0/1385165532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97c815c2a0 0x7f97c80ce930 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009840 tx=0x7f97bc009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:27.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.390+0000 7f97ce89b700 1 -- 172.21.15.79:0/1385165532 shutdown_connections 2023-06-23T18:36:27.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.390+0000 7f97ce89b700 1 --2- 172.21.15.79:0/1385165532 >> v2:172.21.15.155:3300/0 conn(0x7f97c806a5a0 0x7f97c80c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.390+0000 7f97ce89b700 1 --2- 172.21.15.79:0/1385165532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97c815c2a0 0x7f97c80ce930 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.390+0000 7f97ce89b700 1 -- 172.21.15.79:0/1385165532 >> 172.21.15.79:0/1385165532 conn(0x7f97c80cfc60 msgr2=0x7f97c80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:27.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.390+0000 7f97ce89b700 1 -- 172.21.15.79:0/1385165532 shutdown_connections 2023-06-23T18:36:27.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.390+0000 7f97ce89b700 1 -- 172.21.15.79:0/1385165532 wait complete. 2023-06-23T18:36:27.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.391+0000 7f97ce89b700 1 Processor -- start 2023-06-23T18:36:27.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.392+0000 7f97ce89b700 1 -- start start 2023-06-23T18:36:27.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.392+0000 7f97ce89b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97c806a5a0 0x7f97c8146290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.392+0000 7f97ce89b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97c80b1410 0x7f97c8156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.392+0000 7f97ce89b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97c815c2a0 0x7f97c80c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.392+0000 7f97ce89b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c81566e0 con 0x7f97c80b1410 2023-06-23T18:36:27.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.392+0000 7f97ce89b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8155430 con 0x7f97c815c2a0 2023-06-23T18:36:27.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.392+0000 7f97ce89b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8155db0 con 0x7f97c806a5a0 2023-06-23T18:36:27.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.392+0000 7f97c77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97c80b1410 0x7f97c8156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.392+0000 7f97c77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97c80b1410 0x7f97c8156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40318/0 (socket says 172.21.15.79:40318) 2023-06-23T18:36:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.392+0000 7f97c77fe700 1 -- 172.21.15.79:0/3364425427 learned_addr learned my addr 172.21.15.79:0/3364425427 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.393+0000 7f97cce38700 1 --2- 172.21.15.79:0/3364425427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97c815c2a0 0x7f97c80c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.393+0000 7f97c7fff700 1 --2- 172.21.15.79:0/3364425427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97c806a5a0 0x7f97c8146290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.393+0000 7f97c77fe700 1 -- 172.21.15.79:0/3364425427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97c806a5a0 msgr2=0x7f97c8146290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.393+0000 7f97c77fe700 1 --2- 172.21.15.79:0/3364425427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97c806a5a0 0x7f97c8146290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.393+0000 7f97c77fe700 1 -- 172.21.15.79:0/3364425427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97c815c2a0 msgr2=0x7f97c80c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.393+0000 7f97c77fe700 1 --2- 172.21.15.79:0/3364425427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97c815c2a0 0x7f97c80c7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.393+0000 7f97c77fe700 1 -- 172.21.15.79:0/3364425427 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc0094a0 con 0x7f97c80b1410 2023-06-23T18:36:27.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.393+0000 7f97c77fe700 1 --2- 172.21.15.79:0/3364425427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97c80b1410 0x7f97c8156b70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f97b800c520 tx=0x7f97b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:27.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.393+0000 7f97c4ff9700 1 -- 172.21.15.79:0/3364425427 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8004c80 con 0x7f97c80b1410 2023-06-23T18:36:27.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.393+0000 7f97c4ff9700 1 -- 172.21.15.79:0/3364425427 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b8004de0 con 0x7f97c80b1410 2023-06-23T18:36:27.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.393+0000 7f97ce89b700 1 -- 172.21.15.79:0/3364425427 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c8156090 con 0x7f97c80b1410 2023-06-23T18:36:27.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.393+0000 7f97c4ff9700 1 -- 172.21.15.79:0/3364425427 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b80072d0 con 0x7f97c80b1410 2023-06-23T18:36:27.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.393+0000 7f97ce89b700 1 -- 172.21.15.79:0/3364425427 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c80c7d00 con 0x7f97c80b1410 2023-06-23T18:36:27.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.394+0000 7f97ce89b700 1 -- 172.21.15.79:0/3364425427 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c8156850 con 0x7f97c80b1410 2023-06-23T18:36:27.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.394+0000 7f97c4ff9700 1 -- 172.21.15.79:0/3364425427 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f97b8007430 con 0x7f97c80b1410 2023-06-23T18:36:27.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.395+0000 7f97c4ff9700 1 --2- 172.21.15.79:0/3364425427 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f97b406ace0 0x7f97b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.395+0000 7f97c4ff9700 1 -- 172.21.15.79:0/3364425427 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f97b80891b0 con 0x7f97c80b1410 2023-06-23T18:36:27.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.395+0000 7f97c7fff700 1 --2- 172.21.15.79:0/3364425427 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f97b406ace0 0x7f97b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.395+0000 7f97c7fff700 1 --2- 172.21.15.79:0/3364425427 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f97b406ace0 0x7f97b406d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0090b0 tx=0x7f97bc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:27.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.399+0000 7f97c4ff9700 1 -- 172.21.15.79:0/3364425427 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f97b8054260 con 0x7f97c80b1410 2023-06-23T18:36:27.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.585+0000 7f97ce89b700 1 -- 172.21.15.79:0/3364425427 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97c80672c0 con 0x7f97c80b1410 2023-06-23T18:36:27.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.585+0000 7f97c4ff9700 1 -- 172.21.15.79:0/3364425427 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f97b8057880 con 0x7f97c80b1410 2023-06-23T18:36:27.590 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:27.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.587+0000 7f97ce89b700 1 -- 172.21.15.79:0/3364425427 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f97b406ace0 msgr2=0x7f97b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.587+0000 7f97ce89b700 1 --2- 172.21.15.79:0/3364425427 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f97b406ace0 0x7f97b406d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0090b0 tx=0x7f97bc008030 comp rx=0 tx=0).stop 2023-06-23T18:36:27.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.587+0000 7f97ce89b700 1 -- 172.21.15.79:0/3364425427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97c80b1410 msgr2=0x7f97c8156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.587+0000 7f97ce89b700 1 --2- 172.21.15.79:0/3364425427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97c80b1410 0x7f97c8156b70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f97b800c520 tx=0x7f97b800c8e0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.587+0000 7f97ce89b700 1 -- 172.21.15.79:0/3364425427 shutdown_connections 2023-06-23T18:36:27.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.587+0000 7f97ce89b700 1 --2- 172.21.15.79:0/3364425427 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f97b406ace0 0x7f97b406d190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.587+0000 7f97ce89b700 1 --2- 172.21.15.79:0/3364425427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97c815c2a0 0x7f97c80c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.587+0000 7f97ce89b700 1 --2- 172.21.15.79:0/3364425427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97c806a5a0 0x7f97c8146290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.587+0000 7f97ce89b700 1 --2- 172.21.15.79:0/3364425427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97c80b1410 0x7f97c8156b70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.587+0000 7f97ce89b700 1 -- 172.21.15.79:0/3364425427 >> 172.21.15.79:0/3364425427 conn(0x7f97c80cfc60 msgr2=0x7f97c80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:27.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.587+0000 7f97ce89b700 1 -- 172.21.15.79:0/3364425427 shutdown_connections 2023-06-23T18:36:27.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.587+0000 7f97ce89b700 1 -- 172.21.15.79:0/3364425427 wait complete. 2023-06-23T18:36:27.604 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:27.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.721+0000 7f45be92f700 1 Processor -- start 2023-06-23T18:36:27.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.721+0000 7f45be92f700 1 -- start start 2023-06-23T18:36:27.728 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.722+0000 7f45be92f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f45b8156ae0 0x7f45b815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.722+0000 7f45be92f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45b80cc060 0x7f45b80cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.722+0000 7f45be92f700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b80cc9f0 con 0x7f45b8071710 2023-06-23T18:36:27.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.722+0000 7f45be92f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b815daf0 con 0x7f45b8156ae0 2023-06-23T18:36:27.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.722+0000 7f45be92f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b815a560 con 0x7f45b80cc060 2023-06-23T18:36:27.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.722+0000 7f45b7fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f45b8156ae0 0x7f45b815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.722+0000 7f45bcecc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f45b8071710 0x7f45b8071b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43370/0 (socket says 172.21.15.79:43370) 2023-06-23T18:36:27.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.722+0000 7f45bcecc700 1 -- 172.21.15.79:0/1447437864 learned_addr learned my addr 172.21.15.79:0/1447437864 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:27.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.722+0000 7f45b77fe700 1 --2- 172.21.15.79:0/1447437864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45b80cc060 0x7f45b80cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.722+0000 7f45b7fff700 1 -- 172.21.15.79:0/1447437864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45b80cc060 msgr2=0x7f45b80cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:27.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.722+0000 7f45b7fff700 1 --2- 172.21.15.79:0/1447437864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45b80cc060 0x7f45b80cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.722+0000 7f45b7fff700 1 -- 172.21.15.79:0/1447437864 >> v1:172.21.15.79:6789/0 conn(0x7f45b8071710 legacy=0x7f45b8071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.722+0000 7f45b7fff700 1 -- 172.21.15.79:0/1447437864 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b8159950 con 0x7f45b8156ae0 2023-06-23T18:36:27.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.722+0000 7f45b77fe700 1 --2- 172.21.15.79:0/1447437864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45b80cc060 0x7f45b80cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:27.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.723+0000 7f45b7fff700 1 --2- 172.21.15.79:0/1447437864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45b8156ae0 0x7f45b815d5b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009840 tx=0x7f45ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac1613790a7da6a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.723+0000 7f45b6ffd700 1 -- 172.21.15.79:0/1447437864 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac011830 con 0x7f45b8156ae0 2023-06-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.723+0000 7f45b6ffd700 1 -- 172.21.15.79:0/1447437864 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ac011990 con 0x7f45b8156ae0 2023-06-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.723+0000 7f45b6ffd700 1 -- 172.21.15.79:0/1447437864 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac011d20 con 0x7f45b8156ae0 2023-06-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.723+0000 7f45be92f700 1 -- 172.21.15.79:0/1447437864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45b8156ae0 msgr2=0x7f45b815d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.723+0000 7f45be92f700 1 --2- 172.21.15.79:0/1447437864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45b8156ae0 0x7f45b815d5b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009840 tx=0x7f45ac009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.723+0000 7f45be92f700 1 -- 172.21.15.79:0/1447437864 shutdown_connections 2023-06-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.723+0000 7f45be92f700 1 --2- 172.21.15.79:0/1447437864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45b8156ae0 0x7f45b815d5b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.723+0000 7f45be92f700 1 --2- 172.21.15.79:0/1447437864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45b80cc060 0x7f45b80cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.723+0000 7f45be92f700 1 -- 172.21.15.79:0/1447437864 >> 172.21.15.79:0/1447437864 conn(0x7f45b806afa0 msgr2=0x7f45b806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:27.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.723+0000 7f45be92f700 1 -- 172.21.15.79:0/1447437864 shutdown_connections 2023-06-23T18:36:27.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.723+0000 7f45be92f700 1 -- 172.21.15.79:0/1447437864 wait complete. 2023-06-23T18:36:27.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.724+0000 7f45be92f700 1 Processor -- start 2023-06-23T18:36:27.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.725+0000 7f45be92f700 1 -- start start 2023-06-23T18:36:27.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.725+0000 7f45be92f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45b8071710 0x7f45b8155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.725+0000 7f45be92f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b80cc060 0x7f45b8154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.725+0000 7f45be92f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45b8156ae0 0x7f45b80c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.725+0000 7f45be92f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80c34e0 con 0x7f45b8071710 2023-06-23T18:36:27.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.725+0000 7f45be92f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80c37c0 con 0x7f45b80cc060 2023-06-23T18:36:27.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.725+0000 7f45be92f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80c3aa0 con 0x7f45b8156ae0 2023-06-23T18:36:27.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.725+0000 7f45b7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45b8071710 0x7f45b8155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.725+0000 7f45b7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45b8071710 0x7f45b8155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40330/0 (socket says 172.21.15.79:40330) 2023-06-23T18:36:27.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.725+0000 7f45b7fff700 1 -- 172.21.15.79:0/1439479303 learned_addr learned my addr 172.21.15.79:0/1439479303 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:27.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.726+0000 7f45b77fe700 1 --2- 172.21.15.79:0/1439479303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b80cc060 0x7f45b8154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.726+0000 7f45bcecc700 1 --2- 172.21.15.79:0/1439479303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45b8156ae0 0x7f45b80c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.726+0000 7f45b7fff700 1 -- 172.21.15.79:0/1439479303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45b8156ae0 msgr2=0x7f45b80c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.726+0000 7f45b7fff700 1 --2- 172.21.15.79:0/1439479303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45b8156ae0 0x7f45b80c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.726+0000 7f45b7fff700 1 -- 172.21.15.79:0/1439479303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b80cc060 msgr2=0x7f45b8154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.726+0000 7f45b7fff700 1 --2- 172.21.15.79:0/1439479303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b80cc060 0x7f45b8154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.726+0000 7f45b7fff700 1 -- 172.21.15.79:0/1439479303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac0094a0 con 0x7f45b8071710 2023-06-23T18:36:27.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.726+0000 7f45b7fff700 1 --2- 172.21.15.79:0/1439479303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45b8071710 0x7f45b8155460 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f45ac000c00 tx=0x7f45ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:27.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.726+0000 7f45b4ff9700 1 -- 172.21.15.79:0/1439479303 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0038a0 con 0x7f45b8071710 2023-06-23T18:36:27.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.726+0000 7f45b4ff9700 1 -- 172.21.15.79:0/1439479303 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ac01e070 con 0x7f45b8071710 2023-06-23T18:36:27.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.726+0000 7f45b4ff9700 1 -- 172.21.15.79:0/1439479303 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac003c60 con 0x7f45b8071710 2023-06-23T18:36:27.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.726+0000 7f45be92f700 1 -- 172.21.15.79:0/1439479303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b80c3d80 con 0x7f45b8071710 2023-06-23T18:36:27.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.726+0000 7f45be92f700 1 -- 172.21.15.79:0/1439479303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b8155150 con 0x7f45b8071710 2023-06-23T18:36:27.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.727+0000 7f45be92f700 1 -- 172.21.15.79:0/1439479303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b80c42a0 con 0x7f45b8071710 2023-06-23T18:36:27.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.728+0000 7f45b4ff9700 1 -- 172.21.15.79:0/1439479303 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f45ac02d450 con 0x7f45b8071710 2023-06-23T18:36:27.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.728+0000 7f45b4ff9700 1 --2- 172.21.15.79:0/1439479303 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f45a406ace0 0x7f45a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:27.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.728+0000 7f45b4ff9700 1 -- 172.21.15.79:0/1439479303 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f45ac08a760 con 0x7f45b8071710 2023-06-23T18:36:27.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.728+0000 7f45b77fe700 1 --2- 172.21.15.79:0/1439479303 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f45a406ace0 0x7f45a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:27.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.729+0000 7f45b77fe700 1 --2- 172.21.15.79:0/1439479303 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f45a406ace0 0x7f45a406d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f45a8006fd0 tx=0x7f45a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:27.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.732+0000 7f45b4ff9700 1 -- 172.21.15.79:0/1439479303 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f45ac055980 con 0x7f45b8071710 2023-06-23T18:36:27.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.916+0000 7f45be92f700 1 -- 172.21.15.79:0/1439479303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45b81595b0 con 0x7f45b8071710 2023-06-23T18:36:27.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.917+0000 7f45b4ff9700 1 -- 172.21.15.79:0/1439479303 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45ac02c3b0 con 0x7f45b8071710 2023-06-23T18:36:27.922 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:27.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.919+0000 7f45be92f700 1 -- 172.21.15.79:0/1439479303 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f45a406ace0 msgr2=0x7f45a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.919+0000 7f45be92f700 1 --2- 172.21.15.79:0/1439479303 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f45a406ace0 0x7f45a406d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f45a8006fd0 tx=0x7f45a8008040 comp rx=0 tx=0).stop 2023-06-23T18:36:27.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.919+0000 7f45be92f700 1 -- 172.21.15.79:0/1439479303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45b8071710 msgr2=0x7f45b8155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:27.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.919+0000 7f45be92f700 1 --2- 172.21.15.79:0/1439479303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45b8071710 0x7f45b8155460 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f45ac000c00 tx=0x7f45ac004750 comp rx=0 tx=0).stop 2023-06-23T18:36:27.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.919+0000 7f45be92f700 1 -- 172.21.15.79:0/1439479303 shutdown_connections 2023-06-23T18:36:27.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.919+0000 7f45be92f700 1 --2- 172.21.15.79:0/1439479303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45b8156ae0 0x7f45b80c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.919+0000 7f45be92f700 1 --2- 172.21.15.79:0/1439479303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45b8071710 0x7f45b8155460 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.919+0000 7f45be92f700 1 --2- 172.21.15.79:0/1439479303 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f45a406ace0 0x7f45a406d190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.919+0000 7f45be92f700 1 --2- 172.21.15.79:0/1439479303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b80cc060 0x7f45b8154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:27.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.919+0000 7f45be92f700 1 -- 172.21.15.79:0/1439479303 >> 172.21.15.79:0/1439479303 conn(0x7f45b806afa0 msgr2=0x7f45b815c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:27.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.919+0000 7f45be92f700 1 -- 172.21.15.79:0/1439479303 shutdown_connections 2023-06-23T18:36:27.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:27.919+0000 7f45be92f700 1 -- 172.21.15.79:0/1439479303 wait complete. 2023-06-23T18:36:27.936 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:28.059 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.054+0000 7fbb6cfcb700 1 Processor -- start 2023-06-23T18:36:28.059 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.054+0000 7fbb6cfcb700 1 -- start start 2023-06-23T18:36:28.061 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.055+0000 7fbb6cfcb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbb680cb870 0x7fbb68160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.062 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.055+0000 7fbb6cfcb700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb68069e90 0x7fbb6806a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.062 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.055+0000 7fbb6cfcb700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb68160f50 con 0x7fbb680cf1f0 2023-06-23T18:36:28.062 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.055+0000 7fbb6cfcb700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6806a870 con 0x7fbb680cb870 2023-06-23T18:36:28.062 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.055+0000 7fbb6cfcb700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6815cac0 con 0x7fbb68069e90 2023-06-23T18:36:28.062 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.055+0000 7fbb6659c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbb680cb870 0x7fbb68160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.055+0000 7fbb6659c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbb680cb870 0x7fbb68160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40338/0 (socket says 172.21.15.79:40338) 2023-06-23T18:36:28.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.055+0000 7fbb6659c700 1 -- 172.21.15.79:0/2234390839 learned_addr learned my addr 172.21.15.79:0/2234390839 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:28.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.055+0000 7fbb66d9d700 1 --2- 172.21.15.79:0/2234390839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb68069e90 0x7fbb6806a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6659c700 1 -- 172.21.15.79:0/2234390839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb68069e90 msgr2=0x7fbb6806a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:28.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6659c700 1 --2- 172.21.15.79:0/2234390839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb68069e90 0x7fbb6806a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6659c700 1 -- 172.21.15.79:0/2234390839 >> v1:172.21.15.155:6789/0 conn(0x7fbb680cf1f0 legacy=0x7fbb6815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6659c700 1 -- 172.21.15.79:0/2234390839 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb680cd410 con 0x7fbb680cb870 2023-06-23T18:36:28.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6659c700 1 --2- 172.21.15.79:0/2234390839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb680cb870 0x7fbb68160a10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0098f0 tx=0x7fbb5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa252f9cb61f9474 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:28.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6559a700 1 -- 172.21.15.79:0/2234390839 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c00b8a0 con 0x7fbb680cb870 2023-06-23T18:36:28.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6559a700 1 -- 172.21.15.79:0/2234390839 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c00ba00 con 0x7fbb680cb870 2023-06-23T18:36:28.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6559a700 1 -- 172.21.15.79:0/2234390839 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c00bd90 con 0x7fbb680cb870 2023-06-23T18:36:28.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2234390839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb680cb870 msgr2=0x7fbb68160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6cfcb700 1 --2- 172.21.15.79:0/2234390839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb680cb870 0x7fbb68160a10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0098f0 tx=0x7fbb5c009c00 comp rx=0 tx=0).stop 2023-06-23T18:36:28.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2234390839 shutdown_connections 2023-06-23T18:36:28.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6cfcb700 1 --2- 172.21.15.79:0/2234390839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb680cb870 0x7fbb68160a10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6cfcb700 1 --2- 172.21.15.79:0/2234390839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb68069e90 0x7fbb6806a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2234390839 >> 172.21.15.79:0/2234390839 conn(0x7fbb6806e200 msgr2=0x7fbb680cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:28.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.056+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2234390839 shutdown_connections 2023-06-23T18:36:28.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.057+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2234390839 wait complete. 2023-06-23T18:36:28.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.058+0000 7fbb6cfcb700 1 Processor -- start 2023-06-23T18:36:28.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.058+0000 7fbb6cfcb700 1 -- start start 2023-06-23T18:36:28.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.059+0000 7fbb6cfcb700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb68069e90 0x7fbb681555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.059+0000 7fbb6cfcb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb680cb870 0x7fbb6814ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.059+0000 7fbb6cfcb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb680cf1f0 0x7fbb6815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.059+0000 7fbb6cfcb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb681568e0 con 0x7fbb680cf1f0 2023-06-23T18:36:28.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.059+0000 7fbb6cfcb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6814b1d0 con 0x7fbb680cb870 2023-06-23T18:36:28.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.059+0000 7fbb6cfcb700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb68155cd0 con 0x7fbb68069e90 2023-06-23T18:36:28.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.059+0000 7fbb66d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb680cf1f0 0x7fbb6815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.059+0000 7fbb66d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb680cf1f0 0x7fbb6815d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40346/0 (socket says 172.21.15.79:40346) 2023-06-23T18:36:28.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.059+0000 7fbb66d9d700 1 -- 172.21.15.79:0/2697609685 learned_addr learned my addr 172.21.15.79:0/2697609685 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:28.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.059+0000 7fbb6659c700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb68069e90 0x7fbb681555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.059+0000 7fbb65d9b700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb680cb870 0x7fbb6814ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.060+0000 7fbb66d9d700 1 -- 172.21.15.79:0/2697609685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb68069e90 msgr2=0x7fbb681555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.060+0000 7fbb66d9d700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb68069e90 0x7fbb681555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.060+0000 7fbb66d9d700 1 -- 172.21.15.79:0/2697609685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb680cb870 msgr2=0x7fbb6814ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.060+0000 7fbb66d9d700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb680cb870 0x7fbb6814ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.060+0000 7fbb66d9d700 1 -- 172.21.15.79:0/2697609685 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c009550 con 0x7fbb680cf1f0 2023-06-23T18:36:28.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.060+0000 7fbb65d9b700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb680cb870 0x7fbb6814ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:28.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.060+0000 7fbb66d9d700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb680cf1f0 0x7fbb6815d4d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbb60007e60 tx=0x7fbb6000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:28.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.060+0000 7fbb56ffd700 1 -- 172.21.15.79:0/2697609685 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6000ede0 con 0x7fbb680cf1f0 2023-06-23T18:36:28.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.060+0000 7fbb56ffd700 1 -- 172.21.15.79:0/2697609685 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb60014420 con 0x7fbb680cf1f0 2023-06-23T18:36:28.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.060+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2697609685 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb68155fb0 con 0x7fbb680cf1f0 2023-06-23T18:36:28.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.060+0000 7fbb56ffd700 1 -- 172.21.15.79:0/2697609685 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb600147d0 con 0x7fbb680cf1f0 2023-06-23T18:36:28.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.060+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2697609685 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb68156570 con 0x7fbb680cf1f0 2023-06-23T18:36:28.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.061+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2697609685 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb68156a50 con 0x7fbb680cf1f0 2023-06-23T18:36:28.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.061+0000 7fbb56ffd700 1 -- 172.21.15.79:0/2697609685 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbb60014930 con 0x7fbb680cf1f0 2023-06-23T18:36:28.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.062+0000 7fbb56ffd700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbb4806ace0 0x7fbb4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.062+0000 7fbb56ffd700 1 -- 172.21.15.79:0/2697609685 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbb60089c20 con 0x7fbb680cf1f0 2023-06-23T18:36:28.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.062+0000 7fbb6659c700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbb4806ace0 0x7fbb4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.062+0000 7fbb6659c700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbb4806ace0 0x7fbb4806d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009c40 tx=0x7fbb5c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:28.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.066+0000 7fbb56ffd700 1 -- 172.21.15.79:0/2697609685 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbb60058760 con 0x7fbb680cf1f0 2023-06-23T18:36:28.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.251+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2697609685 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb680672c0 con 0x7fbb680cf1f0 2023-06-23T18:36:28.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.252+0000 7fbb56ffd700 1 -- 172.21.15.79:0/2697609685 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbb600582f0 con 0x7fbb680cf1f0 2023-06-23T18:36:28.257 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:28.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.254+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2697609685 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbb4806ace0 msgr2=0x7fbb4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.254+0000 7fbb6cfcb700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbb4806ace0 0x7fbb4806d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009c40 tx=0x7fbb5c008030 comp rx=0 tx=0).stop 2023-06-23T18:36:28.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.254+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2697609685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb680cf1f0 msgr2=0x7fbb6815d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.254+0000 7fbb6cfcb700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb680cf1f0 0x7fbb6815d4d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbb60007e60 tx=0x7fbb6000e3c0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.254+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2697609685 shutdown_connections 2023-06-23T18:36:28.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.254+0000 7fbb6cfcb700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb680cf1f0 0x7fbb6815d4d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.254+0000 7fbb6cfcb700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb68069e90 0x7fbb681555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.254+0000 7fbb6cfcb700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbb4806ace0 0x7fbb4806d190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.254+0000 7fbb6cfcb700 1 --2- 172.21.15.79:0/2697609685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb680cb870 0x7fbb6814ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.254+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2697609685 >> 172.21.15.79:0/2697609685 conn(0x7fbb6806e200 msgr2=0x7fbb680b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:28.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.254+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2697609685 shutdown_connections 2023-06-23T18:36:28.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.254+0000 7fbb6cfcb700 1 -- 172.21.15.79:0/2697609685 wait complete. 2023-06-23T18:36:28.271 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:36:28.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.387+0000 7fd7e5a50700 1 Processor -- start 2023-06-23T18:36:28.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.387+0000 7fd7e5a50700 1 -- start start 2023-06-23T18:36:28.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.387+0000 7fd7e5a50700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd7e0158830 0x7fd7e00c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.387+0000 7fd7e5a50700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e006a5a0 0x7fd7e00c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.387+0000 7fd7e5a50700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e00c3f80 con 0x7fd7e00c5280 2023-06-23T18:36:28.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.387+0000 7fd7e5a50700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e00c4260 con 0x7fd7e0158830 2023-06-23T18:36:28.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.387+0000 7fd7e5a50700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e00c4540 con 0x7fd7e006a5a0 2023-06-23T18:36:28.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.388+0000 7fd7deffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd7e0158830 0x7fd7e00c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.388+0000 7fd7de7fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd7e00c5280 0x7fd7e0069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43394/0 (socket says 172.21.15.79:43394) 2023-06-23T18:36:28.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.388+0000 7fd7de7fc700 1 -- 172.21.15.79:0/2746900495 learned_addr learned my addr 172.21.15.79:0/2746900495 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:28.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.388+0000 7fd7df7fe700 1 --2- 172.21.15.79:0/2746900495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e006a5a0 0x7fd7e00c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.388+0000 7fd7deffd700 1 -- 172.21.15.79:0/2746900495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e006a5a0 msgr2=0x7fd7e00c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:28.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.388+0000 7fd7deffd700 1 --2- 172.21.15.79:0/2746900495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e006a5a0 0x7fd7e00c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.388+0000 7fd7deffd700 1 -- 172.21.15.79:0/2746900495 >> v1:172.21.15.79:6789/0 conn(0x7fd7e00c5280 legacy=0x7fd7e0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.388+0000 7fd7ddffb700 1 -- 172.21.15.79:0/2746900495 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663691086 0 0) 0x7fd7e00c3f80 con 0x7fd7e00c5280 2023-06-23T18:36:28.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.388+0000 7fd7deffd700 1 -- 172.21.15.79:0/2746900495 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e00c4820 con 0x7fd7e0158830 2023-06-23T18:36:28.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.388+0000 7fd7deffd700 1 --2- 172.21.15.79:0/2746900495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e0158830 0x7fd7e00c4d40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009840 tx=0x7fd7c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17680fcc042a4ea1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:28.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.388+0000 7fd7ddffb700 1 -- 172.21.15.79:0/2746900495 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8011830 con 0x7fd7e0158830 2023-06-23T18:36:28.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.388+0000 7fd7ddffb700 1 -- 172.21.15.79:0/2746900495 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c8011990 con 0x7fd7e0158830 2023-06-23T18:36:28.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.388+0000 7fd7ddffb700 1 -- 172.21.15.79:0/2746900495 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8011d20 con 0x7fd7e0158830 2023-06-23T18:36:28.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.389+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2746900495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e0158830 msgr2=0x7fd7e00c4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.389+0000 7fd7e5a50700 1 --2- 172.21.15.79:0/2746900495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e0158830 0x7fd7e00c4d40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009840 tx=0x7fd7c8009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:28.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.389+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2746900495 shutdown_connections 2023-06-23T18:36:28.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.389+0000 7fd7e5a50700 1 --2- 172.21.15.79:0/2746900495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e006a5a0 0x7fd7e00c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.389+0000 7fd7e5a50700 1 --2- 172.21.15.79:0/2746900495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e0158830 0x7fd7e00c4d40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.389+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2746900495 >> 172.21.15.79:0/2746900495 conn(0x7fd7e00cf3d0 msgr2=0x7fd7e00cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:28.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.389+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2746900495 shutdown_connections 2023-06-23T18:36:28.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.389+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2746900495 wait complete. 2023-06-23T18:36:28.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.390+0000 7fd7e5a50700 1 Processor -- start 2023-06-23T18:36:28.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.390+0000 7fd7e5a50700 1 -- start start 2023-06-23T18:36:28.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7e5a50700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e006a5a0 0x7fd7e00b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7e5a50700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e00c5280 0x7fd7e00b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7e5a50700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e0158830 0x7fd7e01e3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7e5a50700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e006d2d0 con 0x7fd7e00c5280 2023-06-23T18:36:28.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7e5a50700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e00b2540 con 0x7fd7e006a5a0 2023-06-23T18:36:28.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7e5a50700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e00b2820 con 0x7fd7e0158830 2023-06-23T18:36:28.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7de7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e00c5280 0x7fd7e00b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7de7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e00c5280 0x7fd7e00b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40358/0 (socket says 172.21.15.79:40358) 2023-06-23T18:36:28.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7de7fc700 1 -- 172.21.15.79:0/2637306256 learned_addr learned my addr 172.21.15.79:0/2637306256 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:28.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7deffd700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e006a5a0 0x7fd7e00b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7df7fe700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e0158830 0x7fd7e01e3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7de7fc700 1 -- 172.21.15.79:0/2637306256 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e0158830 msgr2=0x7fd7e01e3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7de7fc700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e0158830 0x7fd7e01e3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7de7fc700 1 -- 172.21.15.79:0/2637306256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e006a5a0 msgr2=0x7fd7e00b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7de7fc700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e006a5a0 0x7fd7e00b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7de7fc700 1 -- 172.21.15.79:0/2637306256 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c80094a0 con 0x7fd7e00c5280 2023-06-23T18:36:28.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.391+0000 7fd7deffd700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e006a5a0 0x7fd7e00b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:28.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.392+0000 7fd7de7fc700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e00c5280 0x7fd7e00b1d10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00011e0 tx=0x7fd7d000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:28.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.392+0000 7fd7d7fff700 1 -- 172.21.15.79:0/2637306256 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0004c80 con 0x7fd7e00c5280 2023-06-23T18:36:28.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.392+0000 7fd7d7fff700 1 -- 172.21.15.79:0/2637306256 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d0004e00 con 0x7fd7e00c5280 2023-06-23T18:36:28.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.392+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2637306256 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e006cf60 con 0x7fd7e00c5280 2023-06-23T18:36:28.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.392+0000 7fd7d7fff700 1 -- 172.21.15.79:0/2637306256 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0007310 con 0x7fd7e00c5280 2023-06-23T18:36:28.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.392+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2637306256 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e00b2b00 con 0x7fd7e00c5280 2023-06-23T18:36:28.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.393+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2637306256 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e006d480 con 0x7fd7e00c5280 2023-06-23T18:36:28.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.393+0000 7fd7d7fff700 1 -- 172.21.15.79:0/2637306256 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd7d0007470 con 0x7fd7e00c5280 2023-06-23T18:36:28.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.393+0000 7fd7d7fff700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd7c006ace0 0x7fd7c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.394+0000 7fd7d7fff700 1 -- 172.21.15.79:0/2637306256 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd7d0089220 con 0x7fd7e00c5280 2023-06-23T18:36:28.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.394+0000 7fd7deffd700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd7c006ace0 0x7fd7c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.394+0000 7fd7deffd700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd7c006ace0 0x7fd7c006d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80090b0 tx=0x7fd7c801c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:28.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.398+0000 7fd7d7fff700 1 -- 172.21.15.79:0/2637306256 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd7e006d480 con 0x7fd7e00c5280 2023-06-23T18:36:28.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.600+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2637306256 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd7e006d410 con 0x7fd7e00c5280 2023-06-23T18:36:28.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.600+0000 7fd7d7fff700 1 -- 172.21.15.79:0/2637306256 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd7d0054360 con 0x7fd7e00c5280 2023-06-23T18:36:28.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.602+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2637306256 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd7c006ace0 msgr2=0x7fd7c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.602+0000 7fd7e5a50700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd7c006ace0 0x7fd7c006d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80090b0 tx=0x7fd7c801c040 comp rx=0 tx=0).stop 2023-06-23T18:36:28.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.602+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2637306256 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e00c5280 msgr2=0x7fd7e00b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.602+0000 7fd7e5a50700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e00c5280 0x7fd7e00b1d10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00011e0 tx=0x7fd7d000cae0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.602+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2637306256 shutdown_connections 2023-06-23T18:36:28.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.602+0000 7fd7e5a50700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e0158830 0x7fd7e01e3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.602+0000 7fd7e5a50700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd7c006ace0 0x7fd7c006d190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.602+0000 7fd7e5a50700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e006a5a0 0x7fd7e00b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.602+0000 7fd7e5a50700 1 --2- 172.21.15.79:0/2637306256 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7e00c5280 0x7fd7e00b1d10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.602+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2637306256 >> 172.21.15.79:0/2637306256 conn(0x7fd7e00cf3d0 msgr2=0x7fd7e0070230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:28.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.602+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2637306256 shutdown_connections 2023-06-23T18:36:28.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.602+0000 7fd7e5a50700 1 -- 172.21.15.79:0/2637306256 wait complete. 2023-06-23T18:36:28.609 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-23T18:36:28.619 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:36:28.619 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:36:28.636 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:28.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.791+0000 7fb0814c4700 1 Processor -- start 2023-06-23T18:36:28.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.791+0000 7fb0814c4700 1 -- start start 2023-06-23T18:36:28.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.792+0000 7fb0814c4700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb07c0706b0 0x7fb07c0ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.792+0000 7fb0814c4700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07c15d930 con 0x7fb07c070230 2023-06-23T18:36:28.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.792+0000 7fb0814c4700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07c15cd20 con 0x7fb07c159130 2023-06-23T18:36:28.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.792+0000 7fb0814c4700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb07c15d000 con 0x7fb07c0706b0 2023-06-23T18:36:28.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.792+0000 7fb07b7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb07c0706b0 0x7fb07c0ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.792+0000 7fb07a7fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fb07c070230 0x7fb07c0c9c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43406/0 (socket says 172.21.15.79:43406) 2023-06-23T18:36:28.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.792+0000 7fb07a7fc700 1 -- 172.21.15.79:0/2578816392 learned_addr learned my addr 172.21.15.79:0/2578816392 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:28.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.792+0000 7fb07b7fe700 1 -- 172.21.15.79:0/2578816392 >> v1:172.21.15.155:6789/0 conn(0x7fb07c159130 legacy=0x7fb07c0c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.792+0000 7fb07b7fe700 1 -- 172.21.15.79:0/2578816392 >> v1:172.21.15.79:6789/0 conn(0x7fb07c070230 legacy=0x7fb07c0c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.792+0000 7fb079ffb700 1 -- 172.21.15.79:0/2578816392 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1212589918 0 0) 0x7fb07c15d930 con 0x7fb07c070230 2023-06-23T18:36:28.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.792+0000 7fb07b7fe700 1 -- 172.21.15.79:0/2578816392 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c15d2e0 con 0x7fb07c0706b0 2023-06-23T18:36:28.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.792+0000 7fb07b7fe700 1 --2- 172.21.15.79:0/2578816392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c0706b0 0x7fb07c0ca320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb074009d90 tx=0x7fb07400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de390a43ecc13817 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:28.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.793+0000 7fb079ffb700 1 -- 172.21.15.79:0/2578816392 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07400e070 con 0x7fb07c0706b0 2023-06-23T18:36:28.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.793+0000 7fb079ffb700 1 -- 172.21.15.79:0/2578816392 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07400bbd0 con 0x7fb07c0706b0 2023-06-23T18:36:28.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.793+0000 7fb079ffb700 1 -- 172.21.15.79:0/2578816392 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074012430 con 0x7fb07c0706b0 2023-06-23T18:36:28.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.793+0000 7fb0814c4700 1 -- 172.21.15.79:0/2578816392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c0706b0 msgr2=0x7fb07c0ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.793+0000 7fb0814c4700 1 --2- 172.21.15.79:0/2578816392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c0706b0 0x7fb07c0ca320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb074009d90 tx=0x7fb07400b330 comp rx=0 tx=0).stop 2023-06-23T18:36:28.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.793+0000 7fb0814c4700 1 -- 172.21.15.79:0/2578816392 shutdown_connections 2023-06-23T18:36:28.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.793+0000 7fb0814c4700 1 --2- 172.21.15.79:0/2578816392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c0706b0 0x7fb07c0ca320 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.793+0000 7fb0814c4700 1 -- 172.21.15.79:0/2578816392 >> 172.21.15.79:0/2578816392 conn(0x7fb07c0cf3d0 msgr2=0x7fb07c0cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:28.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.793+0000 7fb0814c4700 1 -- 172.21.15.79:0/2578816392 shutdown_connections 2023-06-23T18:36:28.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.793+0000 7fb0814c4700 1 -- 172.21.15.79:0/2578816392 wait complete. 2023-06-23T18:36:28.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.794+0000 7fb0814c4700 1 Processor -- start 2023-06-23T18:36:28.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.795+0000 7fb0814c4700 1 -- start start 2023-06-23T18:36:28.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.795+0000 7fb0814c4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb07c070230 0x7fb07c0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.795+0000 7fb0814c4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb07c0706b0 0x7fb07c1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.795+0000 7fb0814c4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c159130 0x7fb07c14c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.795+0000 7fb0814c4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c14d020 con 0x7fb07c159130 2023-06-23T18:36:28.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.795+0000 7fb0814c4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c069e90 con 0x7fb07c070230 2023-06-23T18:36:28.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.795+0000 7fb0814c4700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c06acf0 con 0x7fb07c0706b0 2023-06-23T18:36:28.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.795+0000 7fb07b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c159130 0x7fb07c14c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.795+0000 7fb07b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c159130 0x7fb07c14c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40374/0 (socket says 172.21.15.79:40374) 2023-06-23T18:36:28.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.795+0000 7fb07b7fe700 1 -- 172.21.15.79:0/3156276665 learned_addr learned my addr 172.21.15.79:0/3156276665 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:28.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.796+0000 7fb07a7fc700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb07c0706b0 0x7fb07c1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.796+0000 7fb07affd700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb07c070230 0x7fb07c0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.796+0000 7fb07b7fe700 1 -- 172.21.15.79:0/3156276665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb07c0706b0 msgr2=0x7fb07c1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.796+0000 7fb07b7fe700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb07c0706b0 0x7fb07c1581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.796+0000 7fb07b7fe700 1 -- 172.21.15.79:0/3156276665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb07c070230 msgr2=0x7fb07c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:28.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.796+0000 7fb07b7fe700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb07c070230 0x7fb07c0af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:28.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.796+0000 7fb07b7fe700 1 -- 172.21.15.79:0/3156276665 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb074009a70 con 0x7fb07c159130 2023-06-23T18:36:28.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.796+0000 7fb07affd700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb07c070230 0x7fb07c0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:28.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.796+0000 7fb07b7fe700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c159130 0x7fb07c14c0d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb074014040 tx=0x7fb074004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:28.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.796+0000 7fb0637fe700 1 -- 172.21.15.79:0/3156276665 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07401b810 con 0x7fb07c159130 2023-06-23T18:36:28.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.796+0000 7fb0637fe700 1 -- 172.21.15.79:0/3156276665 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb074020070 con 0x7fb07c159130 2023-06-23T18:36:28.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.796+0000 7fb0814c4700 1 -- 172.21.15.79:0/3156276665 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c06a0e0 con 0x7fb07c159130 2023-06-23T18:36:28.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.796+0000 7fb0637fe700 1 -- 172.21.15.79:0/3156276665 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07401bbb0 con 0x7fb07c159130 2023-06-23T18:36:28.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.796+0000 7fb0814c4700 1 -- 172.21.15.79:0/3156276665 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c06a470 con 0x7fb07c159130 2023-06-23T18:36:28.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.797+0000 7fb0814c4700 1 -- 172.21.15.79:0/3156276665 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07c1e2ab0 con 0x7fb07c159130 2023-06-23T18:36:28.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.797+0000 7fb0637fe700 1 -- 172.21.15.79:0/3156276665 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb07401bd10 con 0x7fb07c159130 2023-06-23T18:36:28.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.798+0000 7fb0637fe700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb06806ace0 0x7fb06806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:28.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.798+0000 7fb0637fe700 1 -- 172.21.15.79:0/3156276665 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb074022070 con 0x7fb07c159130 2023-06-23T18:36:28.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.798+0000 7fb07affd700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb06806ace0 0x7fb06806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:28.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.798+0000 7fb07affd700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb06806ace0 0x7fb06806d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb07c14d8e0 tx=0x7fb070009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:28.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.802+0000 7fb0637fe700 1 -- 172.21.15.79:0/3156276665 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb07c1e2ab0 con 0x7fb07c159130 2023-06-23T18:36:28.888 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:36:28.891 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:36:29.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.996+0000 7fb0814c4700 1 -- 172.21.15.79:0/3156276665 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb07c1e2ab0 con 0x7fb07c159130 2023-06-23T18:36:29.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.997+0000 7fb0637fe700 1 -- 172.21.15.79:0/3156276665 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7fb07c1e2ab0 con 0x7fb07c159130 2023-06-23T18:36:29.002 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:29.002 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:29.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.999+0000 7fb0814c4700 1 -- 172.21.15.79:0/3156276665 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb06806ace0 msgr2=0x7fb06806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.999+0000 7fb0814c4700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb06806ace0 0x7fb06806d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb07c14d8e0 tx=0x7fb070009040 comp rx=0 tx=0).stop 2023-06-23T18:36:29.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.999+0000 7fb0814c4700 1 -- 172.21.15.79:0/3156276665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c159130 msgr2=0x7fb07c14c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.999+0000 7fb0814c4700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c159130 0x7fb07c14c0d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb074014040 tx=0x7fb074004ad0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.999+0000 7fb0814c4700 1 -- 172.21.15.79:0/3156276665 shutdown_connections 2023-06-23T18:36:29.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.999+0000 7fb0814c4700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c159130 0x7fb07c14c0d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.999+0000 7fb0814c4700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb06806ace0 0x7fb06806d190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.999+0000 7fb0814c4700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb07c070230 0x7fb07c0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.999+0000 7fb0814c4700 1 --2- 172.21.15.79:0/3156276665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb07c0706b0 0x7fb07c1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.999+0000 7fb0814c4700 1 -- 172.21.15.79:0/3156276665 >> 172.21.15.79:0/3156276665 conn(0x7fb07c0cf3d0 msgr2=0x7fb07c06d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:29.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.999+0000 7fb0814c4700 1 -- 172.21.15.79:0/3156276665 shutdown_connections 2023-06-23T18:36:29.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:28.999+0000 7fb0814c4700 1 -- 172.21.15.79:0/3156276665 wait complete. 2023-06-23T18:36:29.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:29.017 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:29.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.136+0000 7fe400c32700 1 Processor -- start 2023-06-23T18:36:29.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.136+0000 7fe400c32700 1 -- start start 2023-06-23T18:36:29.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe400c32700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe3fc0cd440 0x7fe3fc1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe400c32700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3fc0cdc90 con 0x7fe3fc06a5a0 2023-06-23T18:36:29.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe400c32700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3fc15d0f0 con 0x7fe3fc1569c0 2023-06-23T18:36:29.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe400c32700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3fc0cf570 con 0x7fe3fc0cd440 2023-06-23T18:36:29.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3f9d9b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe3fc06a5a0 0x7fe3fc0b12f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43420/0 (socket says 172.21.15.79:43420) 2023-06-23T18:36:29.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3f9d9b700 1 -- 172.21.15.79:0/2458557680 learned_addr learned my addr 172.21.15.79:0/2458557680 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:29.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3fad9d700 1 --2- 172.21.15.79:0/2458557680 >> v2:172.21.15.155:3300/0 conn(0x7fe3fc0cd440 0x7fe3fc1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3f959a700 1 -- 172.21.15.79:0/2458557680 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162737616 0 0) 0x7fe3f0001110 con 0x7fe3fc06a5a0 2023-06-23T18:36:29.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3f959a700 1 -- 172.21.15.79:0/2458557680 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3e40035b0 con 0x7fe3fc06a5a0 2023-06-23T18:36:29.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3f959a700 1 -- 172.21.15.79:0/2458557680 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 890832542 0 0) 0x7fe3e40035b0 con 0x7fe3fc06a5a0 2023-06-23T18:36:29.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3f959a700 1 -- 172.21.15.79:0/2458557680 >> v2:172.21.15.155:3300/0 conn(0x7fe3fc0cd440 msgr2=0x7fe3fc1608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:29.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3f959a700 1 --2- 172.21.15.79:0/2458557680 >> v2:172.21.15.155:3300/0 conn(0x7fe3fc0cd440 0x7fe3fc1608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3f959a700 1 -- 172.21.15.79:0/2458557680 >> v1:172.21.15.155:6789/0 conn(0x7fe3fc1569c0 legacy=0x7fe3fc069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3f959a700 1 -- 172.21.15.79:0/2458557680 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc156d90 con 0x7fe3fc06a5a0 2023-06-23T18:36:29.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3f959a700 1 -- 172.21.15.79:0/2458557680 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fe3f0001800 con 0x7fe3fc06a5a0 2023-06-23T18:36:29.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3f959a700 1 -- 172.21.15.79:0/2458557680 >> v1:172.21.15.79:6789/0 conn(0x7fe3fc06a5a0 legacy=0x7fe3fc0b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3f959a700 1 --2- 172.21.15.79:0/2458557680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3e4003a00 0x7fe3e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3f959a700 1 --2- 172.21.15.79:0/2458557680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e40043f0 0x7fe3e40048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.137+0000 7fe3f959a700 1 --2- 172.21.15.79:0/2458557680 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e40050b0 0x7fe3e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3f959a700 1 -- 172.21.15.79:0/2458557680 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e400c000 con 0x7fe3e40043f0 2023-06-23T18:36:29.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3f959a700 1 -- 172.21.15.79:0/2458557680 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e400c140 con 0x7fe3e4003a00 2023-06-23T18:36:29.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3f959a700 1 -- 172.21.15.79:0/2458557680 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e400c300 con 0x7fe3e40050b0 2023-06-23T18:36:29.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3f9d9b700 1 --2- 172.21.15.79:0/2458557680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e40043f0 0x7fe3e40048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3fa59c700 1 --2- 172.21.15.79:0/2458557680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3e4003a00 0x7fe3e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3fad9d700 1 --2- 172.21.15.79:0/2458557680 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e40050b0 0x7fe3e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3f9d9b700 1 -- 172.21.15.79:0/2458557680 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e40050b0 msgr2=0x7fe3e400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:29.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3f9d9b700 1 --2- 172.21.15.79:0/2458557680 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e40050b0 0x7fe3e400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3f9d9b700 1 -- 172.21.15.79:0/2458557680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3e4003a00 msgr2=0x7fe3e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:29.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3f9d9b700 1 --2- 172.21.15.79:0/2458557680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3e4003a00 0x7fe3e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3f9d9b700 1 -- 172.21.15.79:0/2458557680 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e40035b0 con 0x7fe3e40043f0 2023-06-23T18:36:29.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3fad9d700 1 --2- 172.21.15.79:0/2458557680 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e40050b0 0x7fe3e400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:29.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3fad9d700 1 -- 172.21.15.79:0/2458557680 reap_dead start 2023-06-23T18:36:29.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3f9d9b700 1 --2- 172.21.15.79:0/2458557680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e40043f0 0x7fe3e40048b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000c210 tx=0x7fe3e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffb634122790e25 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:29.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3f959a700 1 -- 172.21.15.79:0/2458557680 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f000cdb0 con 0x7fe3e40043f0 2023-06-23T18:36:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3f959a700 1 -- 172.21.15.79:0/2458557680 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f0005960 con 0x7fe3e40043f0 2023-06-23T18:36:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.138+0000 7fe3f959a700 1 -- 172.21.15.79:0/2458557680 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0005d10 con 0x7fe3e40043f0 2023-06-23T18:36:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.139+0000 7fe400c32700 1 -- 172.21.15.79:0/2458557680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e40043f0 msgr2=0x7fe3e40048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.139+0000 7fe400c32700 1 --2- 172.21.15.79:0/2458557680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e40043f0 0x7fe3e40048b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000c210 tx=0x7fe3e4005880 comp rx=0 tx=0).stop 2023-06-23T18:36:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.139+0000 7fe400c32700 1 -- 172.21.15.79:0/2458557680 shutdown_connections 2023-06-23T18:36:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.139+0000 7fe400c32700 1 --2- 172.21.15.79:0/2458557680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e40043f0 0x7fe3e40048b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.139+0000 7fe400c32700 1 -- 172.21.15.79:0/2458557680 >> 172.21.15.79:0/2458557680 conn(0x7fe3fc06afa0 msgr2=0x7fe3fc06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:29.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.139+0000 7fe400c32700 1 -- 172.21.15.79:0/2458557680 shutdown_connections 2023-06-23T18:36:29.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.139+0000 7fe400c32700 1 -- 172.21.15.79:0/2458557680 wait complete. 2023-06-23T18:36:29.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.140+0000 7fe400c32700 1 Processor -- start 2023-06-23T18:36:29.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.140+0000 7fe400c32700 1 -- start start 2023-06-23T18:36:29.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe400c32700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e40043f0 0x7fe3fc0c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe400c32700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3fc0cb840 0x7fe3fc0ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe400c32700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3fc152e20 0x7fe3fc0b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe400c32700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc0ab8b0 con 0x7fe3fc0cb840 2023-06-23T18:36:29.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe400c32700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc0ae260 con 0x7fe3fc152e20 2023-06-23T18:36:29.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe400c32700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc0adf30 con 0x7fe3e40043f0 2023-06-23T18:36:29.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe3f9d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3fc0cb840 0x7fe3fc0ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe3f9d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3fc0cb840 0x7fe3fc0ae6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40392/0 (socket says 172.21.15.79:40392) 2023-06-23T18:36:29.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe3f9d9b700 1 -- 172.21.15.79:0/3526675265 learned_addr learned my addr 172.21.15.79:0/3526675265 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:29.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe3fad9d700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3fc152e20 0x7fe3fc0b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe3fa59c700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e40043f0 0x7fe3fc0c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe3f9d9b700 1 -- 172.21.15.79:0/3526675265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e40043f0 msgr2=0x7fe3fc0c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe3f9d9b700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e40043f0 0x7fe3fc0c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe3f9d9b700 1 -- 172.21.15.79:0/3526675265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3fc152e20 msgr2=0x7fe3fc0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe3f9d9b700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3fc152e20 0x7fe3fc0b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe3f9d9b700 1 -- 172.21.15.79:0/3526675265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc0abb00 con 0x7fe3fc0cb840 2023-06-23T18:36:29.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe3fad9d700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3fc152e20 0x7fe3fc0b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:29.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe3fa59c700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e40043f0 0x7fe3fc0c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:29.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.141+0000 7fe3f9d9b700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3fc0cb840 0x7fe3fc0ae6f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0001830 tx=0x7fe3f0001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:29.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.142+0000 7fe3eaffd700 1 -- 172.21.15.79:0/3526675265 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0005ce0 con 0x7fe3fc0cb840 2023-06-23T18:36:29.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.142+0000 7fe3eaffd700 1 -- 172.21.15.79:0/3526675265 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f0005e40 con 0x7fe3fc0cb840 2023-06-23T18:36:29.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.142+0000 7fe400c32700 1 -- 172.21.15.79:0/3526675265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc0c5fd0 con 0x7fe3fc0cb840 2023-06-23T18:36:29.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.142+0000 7fe3eaffd700 1 -- 172.21.15.79:0/3526675265 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00072d0 con 0x7fe3fc0cb840 2023-06-23T18:36:29.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.142+0000 7fe400c32700 1 -- 172.21.15.79:0/3526675265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc1532a0 con 0x7fe3fc0cb840 2023-06-23T18:36:29.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.143+0000 7fe400c32700 1 -- 172.21.15.79:0/3526675265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3fc153430 con 0x7fe3fc0cb840 2023-06-23T18:36:29.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.143+0000 7fe3eaffd700 1 -- 172.21.15.79:0/3526675265 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe3f0007430 con 0x7fe3fc0cb840 2023-06-23T18:36:29.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.143+0000 7fe3eaffd700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe3dc06ace0 0x7fe3dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.144+0000 7fe3eaffd700 1 -- 172.21.15.79:0/3526675265 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3f008a720 con 0x7fe3fc0cb840 2023-06-23T18:36:29.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.144+0000 7fe3fa59c700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe3dc06ace0 0x7fe3dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.144+0000 7fe3fa59c700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe3dc06ace0 0x7fe3dc06d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0011a0 tx=0x7fe3ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:29.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.148+0000 7fe3eaffd700 1 -- 172.21.15.79:0/3526675265 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe3fc153430 con 0x7fe3fc0cb840 2023-06-23T18:36:29.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.333+0000 7fe400c32700 1 -- 172.21.15.79:0/3526675265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe3fc0cdc90 con 0x7fe3fc0cb840 2023-06-23T18:36:29.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.334+0000 7fe3eaffd700 1 -- 172.21.15.79:0/3526675265 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe3fc0cdc90 con 0x7fe3fc0cb840 2023-06-23T18:36:29.339 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:29.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.336+0000 7fe400c32700 1 -- 172.21.15.79:0/3526675265 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe3dc06ace0 msgr2=0x7fe3dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.336+0000 7fe400c32700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe3dc06ace0 0x7fe3dc06d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0011a0 tx=0x7fe3ec009040 comp rx=0 tx=0).stop 2023-06-23T18:36:29.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.336+0000 7fe400c32700 1 -- 172.21.15.79:0/3526675265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3fc0cb840 msgr2=0x7fe3fc0ae6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.336+0000 7fe400c32700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3fc0cb840 0x7fe3fc0ae6f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0001830 tx=0x7fe3f0001b90 comp rx=0 tx=0).stop 2023-06-23T18:36:29.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.336+0000 7fe400c32700 1 -- 172.21.15.79:0/3526675265 shutdown_connections 2023-06-23T18:36:29.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.336+0000 7fe400c32700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe3dc06ace0 0x7fe3dc06d190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.336+0000 7fe400c32700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3fc152e20 0x7fe3fc0b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.336+0000 7fe400c32700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e40043f0 0x7fe3fc0c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.336+0000 7fe400c32700 1 --2- 172.21.15.79:0/3526675265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3fc0cb840 0x7fe3fc0ae6f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.336+0000 7fe400c32700 1 -- 172.21.15.79:0/3526675265 >> 172.21.15.79:0/3526675265 conn(0x7fe3fc06afa0 msgr2=0x7fe3fc15d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:29.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.336+0000 7fe400c32700 1 -- 172.21.15.79:0/3526675265 shutdown_connections 2023-06-23T18:36:29.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.336+0000 7fe400c32700 1 -- 172.21.15.79:0/3526675265 wait complete. 2023-06-23T18:36:29.353 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:29.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.471+0000 7f6affd95700 1 Processor -- start 2023-06-23T18:36:29.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.471+0000 7f6affd95700 1 -- start start 2023-06-23T18:36:29.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6affd95700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6af81569c0 0x7f6af81608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6affd95700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6af80cd440 0x7f6af8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6affd95700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af80cdc90 con 0x7f6af815d580 2023-06-23T18:36:29.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6affd95700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af80cf570 con 0x7f6af81569c0 2023-06-23T18:36:29.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6affd95700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af815c8b0 con 0x7f6af80cd440 2023-06-23T18:36:29.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afdb31700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6af81569c0 0x7f6af81608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afdb31700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6af81569c0 0x7f6af81608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40398/0 (socket says 172.21.15.79:40398) 2023-06-23T18:36:29.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afdb31700 1 -- 172.21.15.79:0/2889030717 learned_addr learned my addr 172.21.15.79:0/2889030717 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:29.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afe332700 1 --1- 172.21.15.79:0/2889030717 >> v1:172.21.15.79:6789/0 conn(0x7f6af815d580 0x7f6af80b12f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43440/0 (socket says 172.21.15.79:43440) 2023-06-23T18:36:29.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afdb31700 1 -- 172.21.15.79:0/2889030717 >> v2:172.21.15.155:3300/0 conn(0x7f6af80cd440 msgr2=0x7f6af8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:29.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afd330700 1 --2- 172.21.15.79:0/2889030717 >> v2:172.21.15.155:3300/0 conn(0x7f6af80cd440 0x7f6af8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afdb31700 1 --2- 172.21.15.79:0/2889030717 >> v2:172.21.15.155:3300/0 conn(0x7f6af80cd440 0x7f6af8069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afdb31700 1 -- 172.21.15.79:0/2889030717 >> v1:172.21.15.79:6789/0 conn(0x7f6af815d580 legacy=0x7f6af80b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afcb2f700 1 -- 172.21.15.79:0/2889030717 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192354534 0 0) 0x7f6af0001490 con 0x7f6af815d580 2023-06-23T18:36:29.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afdb31700 1 -- 172.21.15.79:0/2889030717 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af815d0f0 con 0x7f6af81569c0 2023-06-23T18:36:29.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afd330700 1 --2- 172.21.15.79:0/2889030717 >> v2:172.21.15.155:3300/0 conn(0x7f6af80cd440 0x7f6af8069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:29.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afdb31700 1 --2- 172.21.15.79:0/2889030717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6af81569c0 0x7f6af81608b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6af80702d0 tx=0x7f6af4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa0bf61c667d09b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:29.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afcb2f700 1 -- 172.21.15.79:0/2889030717 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af400b870 con 0x7f6af81569c0 2023-06-23T18:36:29.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afcb2f700 1 -- 172.21.15.79:0/2889030717 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af400b9d0 con 0x7f6af81569c0 2023-06-23T18:36:29.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.472+0000 7f6afcb2f700 1 -- 172.21.15.79:0/2889030717 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af400bd60 con 0x7f6af81569c0 2023-06-23T18:36:29.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.473+0000 7f6affd95700 1 -- 172.21.15.79:0/2889030717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6af81569c0 msgr2=0x7f6af81608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.473+0000 7f6affd95700 1 --2- 172.21.15.79:0/2889030717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6af81569c0 0x7f6af81608b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6af80702d0 tx=0x7f6af4009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.473+0000 7f6affd95700 1 -- 172.21.15.79:0/2889030717 shutdown_connections 2023-06-23T18:36:29.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.473+0000 7f6affd95700 1 --2- 172.21.15.79:0/2889030717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6af81569c0 0x7f6af81608b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.473+0000 7f6affd95700 1 --2- 172.21.15.79:0/2889030717 >> v2:172.21.15.155:3300/0 conn(0x7f6af80cd440 0x7f6af8069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.473+0000 7f6affd95700 1 -- 172.21.15.79:0/2889030717 >> 172.21.15.79:0/2889030717 conn(0x7f6af806afa0 msgr2=0x7f6af806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:29.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.473+0000 7f6affd95700 1 -- 172.21.15.79:0/2889030717 shutdown_connections 2023-06-23T18:36:29.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.473+0000 7f6affd95700 1 -- 172.21.15.79:0/2889030717 wait complete. 2023-06-23T18:36:29.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.474+0000 7f6affd95700 1 Processor -- start 2023-06-23T18:36:29.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.474+0000 7f6affd95700 1 -- start start 2023-06-23T18:36:29.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6affd95700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6af80cd440 0x7f6af8148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6affd95700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6af81569c0 0x7f6af814a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6affd95700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6af815d580 0x7f6af814ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6affd95700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80c4f30 con 0x7f6af815d580 2023-06-23T18:36:29.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6affd95700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80ab950 con 0x7f6af81569c0 2023-06-23T18:36:29.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6affd95700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80abc30 con 0x7f6af80cd440 2023-06-23T18:36:29.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6afe332700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6af815d580 0x7f6af814ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6afe332700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6af815d580 0x7f6af814ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40406/0 (socket says 172.21.15.79:40406) 2023-06-23T18:36:29.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6afe332700 1 -- 172.21.15.79:0/3089100077 learned_addr learned my addr 172.21.15.79:0/3089100077 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:29.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6afd330700 1 --2- 172.21.15.79:0/3089100077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6af81569c0 0x7f6af814a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6afdb31700 1 --2- 172.21.15.79:0/3089100077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6af80cd440 0x7f6af8148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6afe332700 1 -- 172.21.15.79:0/3089100077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6af80cd440 msgr2=0x7f6af8148b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6afe332700 1 --2- 172.21.15.79:0/3089100077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6af80cd440 0x7f6af8148b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6afe332700 1 -- 172.21.15.79:0/3089100077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6af81569c0 msgr2=0x7f6af814a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6afe332700 1 --2- 172.21.15.79:0/3089100077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6af81569c0 0x7f6af814a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6afe332700 1 -- 172.21.15.79:0/3089100077 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af4009550 con 0x7f6af815d580 2023-06-23T18:36:29.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.475+0000 7f6afe332700 1 --2- 172.21.15.79:0/3089100077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6af815d580 0x7f6af814ab40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6af000cb60 tx=0x7f6af000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:29.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.476+0000 7f6aee7fc700 1 -- 172.21.15.79:0/3089100077 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af000ab50 con 0x7f6af815d580 2023-06-23T18:36:29.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.476+0000 7f6aee7fc700 1 -- 172.21.15.79:0/3089100077 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af000acb0 con 0x7f6af815d580 2023-06-23T18:36:29.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.476+0000 7f6affd95700 1 -- 172.21.15.79:0/3089100077 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af814b950 con 0x7f6af815d580 2023-06-23T18:36:29.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.476+0000 7f6aee7fc700 1 -- 172.21.15.79:0/3089100077 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0007310 con 0x7f6af815d580 2023-06-23T18:36:29.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.476+0000 7f6affd95700 1 -- 172.21.15.79:0/3089100077 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af80abf10 con 0x7f6af815d580 2023-06-23T18:36:29.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.477+0000 7f6affd95700 1 -- 172.21.15.79:0/3089100077 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af80c50a0 con 0x7f6af815d580 2023-06-23T18:36:29.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.477+0000 7f6aee7fc700 1 -- 172.21.15.79:0/3089100077 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6af0007470 con 0x7f6af815d580 2023-06-23T18:36:29.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.477+0000 7f6aee7fc700 1 --2- 172.21.15.79:0/3089100077 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f6adc06ace0 0x7f6adc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.477+0000 7f6aee7fc700 1 -- 172.21.15.79:0/3089100077 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6af0089110 con 0x7f6af815d580 2023-06-23T18:36:29.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.477+0000 7f6afdb31700 1 --2- 172.21.15.79:0/3089100077 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f6adc06ace0 0x7f6adc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.478+0000 7f6afdb31700 1 --2- 172.21.15.79:0/3089100077 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f6adc06ace0 0x7f6adc06d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6af4000c00 tx=0x7f6af4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:29.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.482+0000 7f6aee7fc700 1 -- 172.21.15.79:0/3089100077 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6af80c50a0 con 0x7f6af815d580 2023-06-23T18:36:29.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.670+0000 7f6affd95700 1 -- 172.21.15.79:0/3089100077 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6af80cdc90 con 0x7f6af815d580 2023-06-23T18:36:29.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.671+0000 7f6aee7fc700 1 -- 172.21.15.79:0/3089100077 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6af0018020 con 0x7f6af815d580 2023-06-23T18:36:29.676 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:29.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.673+0000 7f6affd95700 1 -- 172.21.15.79:0/3089100077 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f6adc06ace0 msgr2=0x7f6adc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.673+0000 7f6affd95700 1 --2- 172.21.15.79:0/3089100077 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f6adc06ace0 0x7f6adc06d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6af4000c00 tx=0x7f6af4008030 comp rx=0 tx=0).stop 2023-06-23T18:36:29.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.673+0000 7f6affd95700 1 -- 172.21.15.79:0/3089100077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6af815d580 msgr2=0x7f6af814ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.673+0000 7f6affd95700 1 --2- 172.21.15.79:0/3089100077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6af815d580 0x7f6af814ab40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6af000cb60 tx=0x7f6af000cf20 comp rx=0 tx=0).stop 2023-06-23T18:36:29.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.673+0000 7f6affd95700 1 -- 172.21.15.79:0/3089100077 shutdown_connections 2023-06-23T18:36:29.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.673+0000 7f6affd95700 1 --2- 172.21.15.79:0/3089100077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6af815d580 0x7f6af814ab40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.673+0000 7f6affd95700 1 --2- 172.21.15.79:0/3089100077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6af80cd440 0x7f6af8148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.673+0000 7f6affd95700 1 --2- 172.21.15.79:0/3089100077 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f6adc06ace0 0x7f6adc06d190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.673+0000 7f6affd95700 1 --2- 172.21.15.79:0/3089100077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6af81569c0 0x7f6af814a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.673+0000 7f6affd95700 1 -- 172.21.15.79:0/3089100077 >> 172.21.15.79:0/3089100077 conn(0x7f6af806afa0 msgr2=0x7f6af815eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:29.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.673+0000 7f6affd95700 1 -- 172.21.15.79:0/3089100077 shutdown_connections 2023-06-23T18:36:29.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.673+0000 7f6affd95700 1 -- 172.21.15.79:0/3089100077 wait complete. 2023-06-23T18:36:29.690 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:29.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.808+0000 7fa35c2ab700 1 Processor -- start 2023-06-23T18:36:29.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.808+0000 7fa35c2ab700 1 -- start start 2023-06-23T18:36:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35c2ab700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa35415c710 0x7fa3540b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35c2ab700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa3540aed90 0x7fa3540b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35c2ab700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa3540cd410 0x7fa35406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35c2ab700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3540ca870 con 0x7fa3540cd410 2023-06-23T18:36:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35c2ab700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35415d620 con 0x7fa3540aed90 2023-06-23T18:36:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35c2ab700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3540cd9d0 con 0x7fa35415c710 2023-06-23T18:36:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35a848700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa3540cd410 0x7fa35406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35a848700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa3540cd410 0x7fa35406eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40412/0 (socket says 172.21.15.79:40412) 2023-06-23T18:36:29.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35a848700 1 -- 172.21.15.79:0/235365097 learned_addr learned my addr 172.21.15.79:0/235365097 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:29.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa359846700 1 --2- 172.21.15.79:0/235365097 >> v2:172.21.15.155:3300/0 conn(0x7fa3540aed90 0x7fa3540b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35a047700 1 --2- 172.21.15.79:0/235365097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa35415c710 0x7fa3540b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35a848700 1 -- 172.21.15.79:0/235365097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa35415c710 msgr2=0x7fa3540b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:29.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35a848700 1 --2- 172.21.15.79:0/235365097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa35415c710 0x7fa3540b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35a848700 1 -- 172.21.15.79:0/235365097 >> v2:172.21.15.155:3300/0 conn(0x7fa3540aed90 msgr2=0x7fa3540b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35a848700 1 --2- 172.21.15.79:0/235365097 >> v2:172.21.15.155:3300/0 conn(0x7fa3540aed90 0x7fa3540b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa35a848700 1 -- 172.21.15.79:0/235365097 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35415ccf0 con 0x7fa3540cd410 2023-06-23T18:36:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.809+0000 7fa359846700 1 --2- 172.21.15.79:0/235365097 >> v2:172.21.15.155:3300/0 conn(0x7fa3540aed90 0x7fa3540b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.810+0000 7fa35a848700 1 --2- 172.21.15.79:0/235365097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3540cd410 0x7fa35406eeb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa3540671e0 tx=0x7fa34c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a729e14ac0eb09 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.810+0000 7fa359045700 1 -- 172.21.15.79:0/235365097 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c00e070 con 0x7fa3540cd410 2023-06-23T18:36:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.810+0000 7fa359045700 1 -- 172.21.15.79:0/235365097 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c00b9c0 con 0x7fa3540cd410 2023-06-23T18:36:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.810+0000 7fa359045700 1 -- 172.21.15.79:0/235365097 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c00bd70 con 0x7fa3540cd410 2023-06-23T18:36:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.810+0000 7fa35c2ab700 1 -- 172.21.15.79:0/235365097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3540cd410 msgr2=0x7fa35406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.810+0000 7fa35c2ab700 1 --2- 172.21.15.79:0/235365097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3540cd410 0x7fa35406eeb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa3540671e0 tx=0x7fa34c00b330 comp rx=0 tx=0).stop 2023-06-23T18:36:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.810+0000 7fa35c2ab700 1 -- 172.21.15.79:0/235365097 shutdown_connections 2023-06-23T18:36:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.810+0000 7fa35c2ab700 1 --2- 172.21.15.79:0/235365097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa35415c710 0x7fa3540b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.810+0000 7fa35c2ab700 1 --2- 172.21.15.79:0/235365097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3540cd410 0x7fa35406eeb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.810+0000 7fa35c2ab700 1 --2- 172.21.15.79:0/235365097 >> v2:172.21.15.155:3300/0 conn(0x7fa3540aed90 0x7fa3540b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.810+0000 7fa35c2ab700 1 -- 172.21.15.79:0/235365097 >> 172.21.15.79:0/235365097 conn(0x7fa35406e200 msgr2=0x7fa3540cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:29.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.810+0000 7fa35c2ab700 1 -- 172.21.15.79:0/235365097 shutdown_connections 2023-06-23T18:36:29.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.810+0000 7fa35c2ab700 1 -- 172.21.15.79:0/235365097 wait complete. 2023-06-23T18:36:29.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.812+0000 7fa35c2ab700 1 Processor -- start 2023-06-23T18:36:29.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.812+0000 7fa35c2ab700 1 -- start start 2023-06-23T18:36:29.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.812+0000 7fa35c2ab700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3540aed90 0x7fa3540c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.812+0000 7fa35c2ab700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3540cd410 0x7fa354156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.812+0000 7fa35c2ab700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa35415c710 0x7fa3540b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.812+0000 7fa35c2ab700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa354158b00 con 0x7fa3540cd410 2023-06-23T18:36:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.812+0000 7fa35c2ab700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35415b6b0 con 0x7fa3540aed90 2023-06-23T18:36:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.812+0000 7fa35c2ab700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3541585b0 con 0x7fa35415c710 2023-06-23T18:36:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa359846700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3540cd410 0x7fa354156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa359846700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3540cd410 0x7fa354156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40414/0 (socket says 172.21.15.79:40414) 2023-06-23T18:36:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa359846700 1 -- 172.21.15.79:0/946910603 learned_addr learned my addr 172.21.15.79:0/946910603 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa35a047700 1 --2- 172.21.15.79:0/946910603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3540aed90 0x7fa3540c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa35a848700 1 --2- 172.21.15.79:0/946910603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa35415c710 0x7fa3540b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa359846700 1 -- 172.21.15.79:0/946910603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa35415c710 msgr2=0x7fa3540b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa359846700 1 --2- 172.21.15.79:0/946910603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa35415c710 0x7fa3540b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa359846700 1 -- 172.21.15.79:0/946910603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3540aed90 msgr2=0x7fa3540c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:29.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa359846700 1 --2- 172.21.15.79:0/946910603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3540aed90 0x7fa3540c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:29.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa359846700 1 -- 172.21.15.79:0/946910603 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34c009b30 con 0x7fa3540cd410 2023-06-23T18:36:29.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa359846700 1 --2- 172.21.15.79:0/946910603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3540cd410 0x7fa354156d50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa34400c820 tx=0x7fa34400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:29.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa34affd700 1 -- 172.21.15.79:0/946910603 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344011800 con 0x7fa3540cd410 2023-06-23T18:36:29.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa34affd700 1 -- 172.21.15.79:0/946910603 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa344011960 con 0x7fa3540cd410 2023-06-23T18:36:29.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa35c2ab700 1 -- 172.21.15.79:0/946910603 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35415b0a0 con 0x7fa3540cd410 2023-06-23T18:36:29.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa34affd700 1 -- 172.21.15.79:0/946910603 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344011d10 con 0x7fa3540cd410 2023-06-23T18:36:29.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.813+0000 7fa35c2ab700 1 -- 172.21.15.79:0/946910603 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35415b340 con 0x7fa3540cd410 2023-06-23T18:36:29.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.814+0000 7fa35c2ab700 1 -- 172.21.15.79:0/946910603 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35415b860 con 0x7fa3540cd410 2023-06-23T18:36:29.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.815+0000 7fa34affd700 1 -- 172.21.15.79:0/946910603 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa344018360 con 0x7fa3540cd410 2023-06-23T18:36:29.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.815+0000 7fa34affd700 1 --2- 172.21.15.79:0/946910603 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa33806ace0 0x7fa33806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:29.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.815+0000 7fa34affd700 1 -- 172.21.15.79:0/946910603 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa344089490 con 0x7fa3540cd410 2023-06-23T18:36:29.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.815+0000 7fa35a047700 1 --2- 172.21.15.79:0/946910603 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa33806ace0 0x7fa33806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:29.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.815+0000 7fa35a047700 1 --2- 172.21.15.79:0/946910603 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa33806ace0 0x7fa33806d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa3540b12f0 tx=0x7fa350008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:29.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:29.819+0000 7fa34affd700 1 -- 172.21.15.79:0/946910603 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa35415b860 con 0x7fa3540cd410 2023-06-23T18:36:30.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.007+0000 7fa35c2ab700 1 -- 172.21.15.79:0/946910603 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa35415b860 con 0x7fa3540cd410 2023-06-23T18:36:30.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.007+0000 7fa34affd700 1 -- 172.21.15.79:0/946910603 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa35415b860 con 0x7fa3540cd410 2023-06-23T18:36:30.012 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:30.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.009+0000 7fa35c2ab700 1 -- 172.21.15.79:0/946910603 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa33806ace0 msgr2=0x7fa33806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:30.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.009+0000 7fa35c2ab700 1 --2- 172.21.15.79:0/946910603 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa33806ace0 0x7fa33806d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa3540b12f0 tx=0x7fa350008040 comp rx=0 tx=0).stop 2023-06-23T18:36:30.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.009+0000 7fa35c2ab700 1 -- 172.21.15.79:0/946910603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3540cd410 msgr2=0x7fa354156d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:30.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.009+0000 7fa35c2ab700 1 --2- 172.21.15.79:0/946910603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3540cd410 0x7fa354156d50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa34400c820 tx=0x7fa34400cbe0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.009+0000 7fa35c2ab700 1 -- 172.21.15.79:0/946910603 shutdown_connections 2023-06-23T18:36:30.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.009+0000 7fa35c2ab700 1 --2- 172.21.15.79:0/946910603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa35415c710 0x7fa3540b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.009+0000 7fa35c2ab700 1 --2- 172.21.15.79:0/946910603 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa33806ace0 0x7fa33806d190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.009+0000 7fa35c2ab700 1 --2- 172.21.15.79:0/946910603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3540aed90 0x7fa3540c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.009+0000 7fa35c2ab700 1 --2- 172.21.15.79:0/946910603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3540cd410 0x7fa354156d50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.009+0000 7fa35c2ab700 1 -- 172.21.15.79:0/946910603 >> 172.21.15.79:0/946910603 conn(0x7fa35406e200 msgr2=0x7fa3540b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:30.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.009+0000 7fa35c2ab700 1 -- 172.21.15.79:0/946910603 shutdown_connections 2023-06-23T18:36:30.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.009+0000 7fa35c2ab700 1 -- 172.21.15.79:0/946910603 wait complete. 2023-06-23T18:36:30.026 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:30.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.143+0000 7fb1fcdd4700 1 Processor -- start 2023-06-23T18:36:30.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.143+0000 7fb1fcdd4700 1 -- start start 2023-06-23T18:36:30.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.144+0000 7fb1fcdd4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f80cd750 0x7fb1f8159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.144+0000 7fb1fcdd4700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb1f8158db0 0x7fb1f815c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.144+0000 7fb1fcdd4700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f815d3c0 con 0x7fb1f815be70 2023-06-23T18:36:30.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.144+0000 7fb1fcdd4700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f815c240 con 0x7fb1f8158db0 2023-06-23T18:36:30.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.144+0000 7fb1fcdd4700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f8159210 con 0x7fb1f80cd750 2023-06-23T18:36:30.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.144+0000 7fb1f6d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb1f8158db0 0x7fb1f815c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.144+0000 7fb1f6d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb1f8158db0 0x7fb1f815c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40426/0 (socket says 172.21.15.79:40426) 2023-06-23T18:36:30.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.144+0000 7fb1f6d9d700 1 -- 172.21.15.79:0/3821800421 learned_addr learned my addr 172.21.15.79:0/3821800421 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:30.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.145+0000 7fb1f5d9b700 1 --2- 172.21.15.79:0/3821800421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f80cd750 0x7fb1f8159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.145+0000 7fb1f6d9d700 1 -- 172.21.15.79:0/3821800421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f80cd750 msgr2=0x7fb1f8159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:30.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.145+0000 7fb1f6d9d700 1 --2- 172.21.15.79:0/3821800421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f80cd750 0x7fb1f8159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.145+0000 7fb1f6d9d700 1 -- 172.21.15.79:0/3821800421 >> v1:172.21.15.155:6789/0 conn(0x7fb1f815be70 legacy=0x7fb1f80cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:30.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.145+0000 7fb1f6d9d700 1 -- 172.21.15.79:0/3821800421 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f806a0b0 con 0x7fb1f8158db0 2023-06-23T18:36:30.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.145+0000 7fb1f6d9d700 1 --2- 172.21.15.79:0/3821800421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1f8158db0 0x7fb1f815c420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000d150 tx=0x7fb1f000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1ed1ef1259cec8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:30.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.145+0000 7fb1f559a700 1 -- 172.21.15.79:0/3821800421 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0010070 con 0x7fb1f8158db0 2023-06-23T18:36:30.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.145+0000 7fb1f559a700 1 -- 172.21.15.79:0/3821800421 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f000dd80 con 0x7fb1f8158db0 2023-06-23T18:36:30.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.145+0000 7fb1f559a700 1 -- 172.21.15.79:0/3821800421 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f000b310 con 0x7fb1f8158db0 2023-06-23T18:36:30.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.145+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/3821800421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1f8158db0 msgr2=0x7fb1f815c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:30.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.145+0000 7fb1fcdd4700 1 --2- 172.21.15.79:0/3821800421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1f8158db0 0x7fb1f815c420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000d150 tx=0x7fb1f000d460 comp rx=0 tx=0).stop 2023-06-23T18:36:30.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.145+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/3821800421 shutdown_connections 2023-06-23T18:36:30.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.145+0000 7fb1fcdd4700 1 --2- 172.21.15.79:0/3821800421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1f8158db0 0x7fb1f815c420 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.145+0000 7fb1fcdd4700 1 --2- 172.21.15.79:0/3821800421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f80cd750 0x7fb1f8159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.146+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/3821800421 >> 172.21.15.79:0/3821800421 conn(0x7fb1f80cfc60 msgr2=0x7fb1f80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:30.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.146+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/3821800421 shutdown_connections 2023-06-23T18:36:30.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.146+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/3821800421 wait complete. 2023-06-23T18:36:30.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.147+0000 7fb1fcdd4700 1 Processor -- start 2023-06-23T18:36:30.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.147+0000 7fb1fcdd4700 1 -- start start 2023-06-23T18:36:30.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1fcdd4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1f80cd750 0x7fb1f81555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1fcdd4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f8158db0 0x7fb1f80c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1fcdd4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f815be70 0x7fb1f80afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1fcdd4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f80c8a40 con 0x7fb1f80cd750 2023-06-23T18:36:30.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1fcdd4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f80c7e30 con 0x7fb1f8158db0 2023-06-23T18:36:30.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1fcdd4700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f80c8110 con 0x7fb1f815be70 2023-06-23T18:36:30.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1f659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1f80cd750 0x7fb1f81555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1f659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1f80cd750 0x7fb1f81555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40428/0 (socket says 172.21.15.79:40428) 2023-06-23T18:36:30.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1f659c700 1 -- 172.21.15.79:0/1371240396 learned_addr learned my addr 172.21.15.79:0/1371240396 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:30.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1f5d9b700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f8158db0 0x7fb1f80c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1f6d9d700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f815be70 0x7fb1f80afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1f659c700 1 -- 172.21.15.79:0/1371240396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f815be70 msgr2=0x7fb1f80afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:30.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1f659c700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f815be70 0x7fb1f80afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1f659c700 1 -- 172.21.15.79:0/1371240396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f8158db0 msgr2=0x7fb1f80c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:30.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1f659c700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f8158db0 0x7fb1f80c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1f659c700 1 -- 172.21.15.79:0/1371240396 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f000bd40 con 0x7fb1f80cd750 2023-06-23T18:36:30.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1f5d9b700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f8158db0 0x7fb1f80c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:30.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1f659c700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1f80cd750 0x7fb1f81555b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00c4d0 tx=0x7fb1ec00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:30.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1e6ffd700 1 -- 172.21.15.79:0/1371240396 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec011790 con 0x7fb1f80cd750 2023-06-23T18:36:30.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1e6ffd700 1 -- 172.21.15.79:0/1371240396 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec0118f0 con 0x7fb1f80cd750 2023-06-23T18:36:30.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1e6ffd700 1 -- 172.21.15.79:0/1371240396 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec011ca0 con 0x7fb1f80cd750 2023-06-23T18:36:30.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/1371240396 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f80c83f0 con 0x7fb1f80cd750 2023-06-23T18:36:30.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.148+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/1371240396 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f8157a80 con 0x7fb1f80cd750 2023-06-23T18:36:30.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.149+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/1371240396 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f80672c0 con 0x7fb1f80cd750 2023-06-23T18:36:30.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.150+0000 7fb1e6ffd700 1 -- 172.21.15.79:0/1371240396 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb1ec01c460 con 0x7fb1f80cd750 2023-06-23T18:36:30.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.150+0000 7fb1e6ffd700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb1d806ace0 0x7fb1d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.150+0000 7fb1f5d9b700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb1d806ace0 0x7fb1d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.150+0000 7fb1e6ffd700 1 -- 172.21.15.79:0/1371240396 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb1ec0892d0 con 0x7fb1f80cd750 2023-06-23T18:36:30.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.151+0000 7fb1f5d9b700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb1d806ace0 0x7fb1d806d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8006fd0 tx=0x7fb1e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:30.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.155+0000 7fb1e6ffd700 1 -- 172.21.15.79:0/1371240396 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb1ec054380 con 0x7fb1f80cd750 2023-06-23T18:36:30.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.341+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/1371240396 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1f815aa40 con 0x7fb1f80cd750 2023-06-23T18:36:30.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.341+0000 7fb1e6ffd700 1 -- 172.21.15.79:0/1371240396 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb1ec0579a0 con 0x7fb1f80cd750 2023-06-23T18:36:30.346 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:30.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.343+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/1371240396 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb1d806ace0 msgr2=0x7fb1d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:30.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.343+0000 7fb1fcdd4700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb1d806ace0 0x7fb1d806d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8006fd0 tx=0x7fb1e8008040 comp rx=0 tx=0).stop 2023-06-23T18:36:30.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.343+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/1371240396 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1f80cd750 msgr2=0x7fb1f81555b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:30.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.343+0000 7fb1fcdd4700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1f80cd750 0x7fb1f81555b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00c4d0 tx=0x7fb1ec00c890 comp rx=0 tx=0).stop 2023-06-23T18:36:30.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.343+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/1371240396 shutdown_connections 2023-06-23T18:36:30.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.343+0000 7fb1fcdd4700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f815be70 0x7fb1f80afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.343+0000 7fb1fcdd4700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1f80cd750 0x7fb1f81555b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.343+0000 7fb1fcdd4700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb1d806ace0 0x7fb1d806d190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.343+0000 7fb1fcdd4700 1 --2- 172.21.15.79:0/1371240396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f8158db0 0x7fb1f80c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.343+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/1371240396 >> 172.21.15.79:0/1371240396 conn(0x7fb1f80cfc60 msgr2=0x7fb1f81595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:30.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.344+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/1371240396 shutdown_connections 2023-06-23T18:36:30.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.344+0000 7fb1fcdd4700 1 -- 172.21.15.79:0/1371240396 wait complete. 2023-06-23T18:36:30.361 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:36:30.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.476+0000 7fd4e0f7c700 1 Processor -- start 2023-06-23T18:36:30.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.476+0000 7fd4e0f7c700 1 -- start start 2023-06-23T18:36:30.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.477+0000 7fd4e0f7c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd4dc15fa60 0x7fd4dc0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.477+0000 7fd4e0f7c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc1609f0 0x7fd4dc0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.477+0000 7fd4e0f7c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4dc15fe30 con 0x7fd4dc0b17d0 2023-06-23T18:36:30.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.477+0000 7fd4e0f7c700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4dc160e30 con 0x7fd4dc15fa60 2023-06-23T18:36:30.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.477+0000 7fd4e0f7c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc06ce10 con 0x7fd4dc1609f0 2023-06-23T18:36:30.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.477+0000 7fd4d9d9b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd4dc0b17d0 0x7fd4dc1602e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43482/0 (socket says 172.21.15.79:43482) 2023-06-23T18:36:30.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.477+0000 7fd4d9d9b700 1 -- 172.21.15.79:0/2740320550 learned_addr learned my addr 172.21.15.79:0/2740320550 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:30.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.477+0000 7fd4da59c700 1 --2- 172.21.15.79:0/2740320550 >> v2:172.21.15.155:3300/0 conn(0x7fd4dc15fa60 0x7fd4dc0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.477+0000 7fd4dad9d700 1 --2- 172.21.15.79:0/2740320550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc1609f0 0x7fd4dc0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.477+0000 7fd4d959a700 1 -- 172.21.15.79:0/2740320550 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079685268 0 0) 0x7fd4cc001110 con 0x7fd4dc0b17d0 2023-06-23T18:36:30.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.477+0000 7fd4d959a700 1 -- 172.21.15.79:0/2740320550 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4c40035b0 con 0x7fd4dc0b17d0 2023-06-23T18:36:30.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 -- 172.21.15.79:0/2740320550 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2498865689 0 0) 0x7fd4c40035b0 con 0x7fd4dc0b17d0 2023-06-23T18:36:30.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 -- 172.21.15.79:0/2740320550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc1609f0 msgr2=0x7fd4dc0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:30.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 --2- 172.21.15.79:0/2740320550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc1609f0 0x7fd4dc0cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 -- 172.21.15.79:0/2740320550 >> v2:172.21.15.155:3300/0 conn(0x7fd4dc15fa60 msgr2=0x7fd4dc0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:30.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 --2- 172.21.15.79:0/2740320550 >> v2:172.21.15.155:3300/0 conn(0x7fd4dc15fa60 0x7fd4dc0ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 -- 172.21.15.79:0/2740320550 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc0b0a10 con 0x7fd4dc0b17d0 2023-06-23T18:36:30.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 -- 172.21.15.79:0/2740320550 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fd4cc001800 con 0x7fd4dc0b17d0 2023-06-23T18:36:30.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 -- 172.21.15.79:0/2740320550 >> v1:172.21.15.79:6789/0 conn(0x7fd4dc0b17d0 legacy=0x7fd4dc1602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:30.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 --2- 172.21.15.79:0/2740320550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4003a00 0x7fd4c4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 --2- 172.21.15.79:0/2740320550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c40043e0 0x7fd4c40048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 --2- 172.21.15.79:0/2740320550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4c4004f20 0x7fd4c400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 -- 172.21.15.79:0/2740320550 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc160e30 con 0x7fd4c4004f20 2023-06-23T18:36:30.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 -- 172.21.15.79:0/2740320550 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc06ce10 con 0x7fd4c40043e0 2023-06-23T18:36:30.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 -- 172.21.15.79:0/2740320550 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c400bf50 con 0x7fd4c4003a00 2023-06-23T18:36:30.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4da59c700 1 --2- 172.21.15.79:0/2740320550 >> v2:172.21.15.155:3300/0 conn(0x7fd4dc15fa60 0x7fd4dc0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:30.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4dad9d700 1 --2- 172.21.15.79:0/2740320550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc1609f0 0x7fd4dc0cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4dad9d700 1 --2- 172.21.15.79:0/2740320550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4c4004f20 0x7fd4c400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d9d9b700 1 --2- 172.21.15.79:0/2740320550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c40043e0 0x7fd4c40048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4da59c700 1 --2- 172.21.15.79:0/2740320550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4003a00 0x7fd4c4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4dad9d700 1 -- 172.21.15.79:0/2740320550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4003a00 msgr2=0x7fd4c4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4dad9d700 1 --2- 172.21.15.79:0/2740320550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c4003a00 0x7fd4c4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4dad9d700 1 -- 172.21.15.79:0/2740320550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c40043e0 msgr2=0x7fd4c40048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4dad9d700 1 --2- 172.21.15.79:0/2740320550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c40043e0 0x7fd4c40048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4dad9d700 1 -- 172.21.15.79:0/2740320550 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c40035b0 con 0x7fd4c4004f20 2023-06-23T18:36:30.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4dad9d700 1 --2- 172.21.15.79:0/2740320550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4c4004f20 0x7fd4c400ba10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc160dc0 tx=0x7fd4c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f331dabccf64b6f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:30.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.478+0000 7fd4d959a700 1 -- 172.21.15.79:0/2740320550 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4003c20 con 0x7fd4c4004f20 2023-06-23T18:36:30.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.479+0000 7fd4d959a700 1 -- 172.21.15.79:0/2740320550 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d400f040 con 0x7fd4c4004f20 2023-06-23T18:36:30.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.479+0000 7fd4d959a700 1 -- 172.21.15.79:0/2740320550 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400ea60 con 0x7fd4c4004f20 2023-06-23T18:36:30.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.479+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/2740320550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4c4004f20 msgr2=0x7fd4c400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:30.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.479+0000 7fd4e0f7c700 1 --2- 172.21.15.79:0/2740320550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4c4004f20 0x7fd4c400ba10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc160dc0 tx=0x7fd4c4005880 comp rx=0 tx=0).stop 2023-06-23T18:36:30.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.479+0000 7fd4dad9d700 1 -- 172.21.15.79:0/2740320550 reap_dead start 2023-06-23T18:36:30.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.479+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/2740320550 shutdown_connections 2023-06-23T18:36:30.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.479+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/2740320550 >> 172.21.15.79:0/2740320550 conn(0x7fd4dc089fc0 msgr2=0x7fd4dc099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:30.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.479+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/2740320550 shutdown_connections 2023-06-23T18:36:30.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.479+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/2740320550 wait complete. 2023-06-23T18:36:30.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.480+0000 7fd4e0f7c700 1 Processor -- start 2023-06-23T18:36:30.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.481+0000 7fd4e0f7c700 1 -- start start 2023-06-23T18:36:30.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.481+0000 7fd4e0f7c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4c4004f20 0x7fd4dc0b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.481+0000 7fd4e0f7c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4dc0c8eb0 0x7fd4dc15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.481+0000 7fd4e0f7c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc15c550 0x7fd4dc0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.481+0000 7fd4e0f7c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc15ce40 con 0x7fd4c4004f20 2023-06-23T18:36:30.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.481+0000 7fd4e0f7c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0c8cb0 con 0x7fd4dc0c8eb0 2023-06-23T18:36:30.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.481+0000 7fd4e0f7c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0c89d0 con 0x7fd4dc15c550 2023-06-23T18:36:30.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.481+0000 7fd4da59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4c4004f20 0x7fd4dc0b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.481+0000 7fd4da59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4c4004f20 0x7fd4dc0b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40448/0 (socket says 172.21.15.79:40448) 2023-06-23T18:36:30.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.481+0000 7fd4da59c700 1 -- 172.21.15.79:0/1545380266 learned_addr learned my addr 172.21.15.79:0/1545380266 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:30.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.481+0000 7fd4d9d9b700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4dc0c8eb0 0x7fd4dc15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.481+0000 7fd4dad9d700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc15c550 0x7fd4dc0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.482+0000 7fd4da59c700 1 -- 172.21.15.79:0/1545380266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc15c550 msgr2=0x7fd4dc0b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:30.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.482+0000 7fd4da59c700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc15c550 0x7fd4dc0b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.482+0000 7fd4da59c700 1 -- 172.21.15.79:0/1545380266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4dc0c8eb0 msgr2=0x7fd4dc15bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:30.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.482+0000 7fd4da59c700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4dc0c8eb0 0x7fd4dc15bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.482+0000 7fd4da59c700 1 -- 172.21.15.79:0/1545380266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc0b3860 con 0x7fd4c4004f20 2023-06-23T18:36:30.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.482+0000 7fd4dad9d700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc15c550 0x7fd4dc0b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:30.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.482+0000 7fd4da59c700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4c4004f20 0x7fd4dc0b44b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0011930 tx=0x7fd4d0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:30.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.482+0000 7fd4caffd700 1 -- 172.21.15.79:0/1545380266 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d000f8d0 con 0x7fd4c4004f20 2023-06-23T18:36:30.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.482+0000 7fd4caffd700 1 -- 172.21.15.79:0/1545380266 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d000fa30 con 0x7fd4c4004f20 2023-06-23T18:36:30.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.482+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/1545380266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc0b3b40 con 0x7fd4c4004f20 2023-06-23T18:36:30.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.482+0000 7fd4caffd700 1 -- 172.21.15.79:0/1545380266 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d000fda0 con 0x7fd4c4004f20 2023-06-23T18:36:30.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.482+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/1545380266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc0b6380 con 0x7fd4c4004f20 2023-06-23T18:36:30.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.483+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/1545380266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc06a7b0 con 0x7fd4c4004f20 2023-06-23T18:36:30.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.483+0000 7fd4caffd700 1 -- 172.21.15.79:0/1545380266 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd4d0015460 con 0x7fd4c4004f20 2023-06-23T18:36:30.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.484+0000 7fd4caffd700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd4bc06ace0 0x7fd4bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:30.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.484+0000 7fd4caffd700 1 -- 172.21.15.79:0/1545380266 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd4d0089e30 con 0x7fd4c4004f20 2023-06-23T18:36:30.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.484+0000 7fd4d9d9b700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd4bc06ace0 0x7fd4bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:30.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.484+0000 7fd4d9d9b700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd4bc06ace0 0x7fd4bc06d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc001050 tx=0x7fd4cc007b00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:30.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.488+0000 7fd4caffd700 1 -- 172.21.15.79:0/1545380266 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd4d0054ee0 con 0x7fd4c4004f20 2023-06-23T18:36:30.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.689+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/1545380266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd4dc0b4300 con 0x7fd4c4004f20 2023-06-23T18:36:30.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.690+0000 7fd4caffd700 1 -- 172.21.15.79:0/1545380266 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd4d0058500 con 0x7fd4c4004f20 2023-06-23T18:36:30.696 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.691+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/1545380266 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd4bc06ace0 msgr2=0x7fd4bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:30.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.691+0000 7fd4e0f7c700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd4bc06ace0 0x7fd4bc06d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc001050 tx=0x7fd4cc007b00 comp rx=0 tx=0).stop 2023-06-23T18:36:30.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.691+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/1545380266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4c4004f20 msgr2=0x7fd4dc0b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:30.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.691+0000 7fd4e0f7c700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4c4004f20 0x7fd4dc0b44b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0011930 tx=0x7fd4d0011c40 comp rx=0 tx=0).stop 2023-06-23T18:36:30.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.692+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/1545380266 shutdown_connections 2023-06-23T18:36:30.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.692+0000 7fd4e0f7c700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4dc15c550 0x7fd4dc0b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.692+0000 7fd4e0f7c700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4c4004f20 0x7fd4dc0b44b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.692+0000 7fd4e0f7c700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd4bc06ace0 0x7fd4bc06d190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.698 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.692+0000 7fd4e0f7c700 1 --2- 172.21.15.79:0/1545380266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4dc0c8eb0 0x7fd4dc15bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:30.698 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.692+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/1545380266 >> 172.21.15.79:0/1545380266 conn(0x7fd4dc089fc0 msgr2=0x7fd4dc1602e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:30.698 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.692+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/1545380266 shutdown_connections 2023-06-23T18:36:30.698 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:30.692+0000 7fd4e0f7c700 1 -- 172.21.15.79:0/1545380266 wait complete. 2023-06-23T18:36:30.698 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-23T18:36:30.708 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi079.front.sepia.ceph.com... 2023-06-23T18:36:30.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:30.709 DEBUG:teuthology.orchestra.run.smithi079:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-06-23T18:36:30.749 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:36:30.749 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:36:30.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:30.750 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-06-23T18:36:30.811 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:36:30.812 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:36:30.833 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:36:30.833 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:36:30.893 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:31.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.010+0000 7ffa10413700 1 Processor -- start 2023-06-23T18:36:31.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.010+0000 7ffa10413700 1 -- start start 2023-06-23T18:36:31.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa10413700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ffa08071490 0x7ffa08071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa10413700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa081608f0 con 0x7ffa080b1270 2023-06-23T18:36:31.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa10413700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa08160e20 con 0x7ffa080aee30 2023-06-23T18:36:31.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa10413700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0806acb0 con 0x7ffa08071490 2023-06-23T18:36:31.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa0d9ae700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ffa080b1270 0x7ffa08069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43498/0 (socket says 172.21.15.79:43498) 2023-06-23T18:36:31.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa0d9ae700 1 -- 172.21.15.79:0/2762395962 learned_addr learned my addr 172.21.15.79:0/2762395962 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:31.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa0d1ad700 1 -- 172.21.15.79:0/2762395962 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640461964 0 0) 0x7ffa081608f0 con 0x7ffa080b1270 2023-06-23T18:36:31.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa0d1ad700 1 -- 172.21.15.79:0/2762395962 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9f40035b0 con 0x7ffa080b1270 2023-06-23T18:36:31.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa0e1af700 1 --2- 172.21.15.79:0/2762395962 >> v2:172.21.15.155:3300/0 conn(0x7ffa08071490 0x7ffa08071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa0d1ad700 1 -- 172.21.15.79:0/2762395962 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3293755482 0 0) 0x7ff9f40035b0 con 0x7ffa080b1270 2023-06-23T18:36:31.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa0d1ad700 1 -- 172.21.15.79:0/2762395962 >> v2:172.21.15.155:3300/0 conn(0x7ffa08071490 msgr2=0x7ffa08071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:31.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa0d1ad700 1 --2- 172.21.15.79:0/2762395962 >> v2:172.21.15.155:3300/0 conn(0x7ffa08071490 0x7ffa08071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa0d1ad700 1 -- 172.21.15.79:0/2762395962 >> v1:172.21.15.155:6789/0 conn(0x7ffa080aee30 legacy=0x7ffa0806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa0d1ad700 1 -- 172.21.15.79:0/2762395962 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa080cd4f0 con 0x7ffa080b1270 2023-06-23T18:36:31.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa0d1ad700 1 -- 172.21.15.79:0/2762395962 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7ff9f8001670 con 0x7ffa080b1270 2023-06-23T18:36:31.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.011+0000 7ffa0d1ad700 1 -- 172.21.15.79:0/2762395962 >> v1:172.21.15.79:6789/0 conn(0x7ffa080b1270 legacy=0x7ffa08069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0d1ad700 1 --2- 172.21.15.79:0/2762395962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9f4003a00 0x7ff9f4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0d1ad700 1 --2- 172.21.15.79:0/2762395962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9f4004850 0x7ff9f4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0d1ad700 1 --2- 172.21.15.79:0/2762395962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9f4005200 0x7ff9f400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0d1ad700 1 -- 172.21.15.79:0/2762395962 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0806acb0 con 0x7ff9f4003a00 2023-06-23T18:36:31.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0d1ad700 1 -- 172.21.15.79:0/2762395962 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f400c110 con 0x7ff9f4004850 2023-06-23T18:36:31.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0d1ad700 1 -- 172.21.15.79:0/2762395962 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f400c2d0 con 0x7ff9f4005200 2023-06-23T18:36:31.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0e1af700 1 --2- 172.21.15.79:0/2762395962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9f4003a00 0x7ff9f4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0d9ae700 1 --2- 172.21.15.79:0/2762395962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9f4004850 0x7ff9f4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0e9b0700 1 --2- 172.21.15.79:0/2762395962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9f4005200 0x7ff9f400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0e1af700 1 -- 172.21.15.79:0/2762395962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9f4005200 msgr2=0x7ff9f400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:31.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0e1af700 1 --2- 172.21.15.79:0/2762395962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9f4005200 0x7ff9f400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0e1af700 1 -- 172.21.15.79:0/2762395962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9f4004850 msgr2=0x7ff9f4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:31.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0e1af700 1 --2- 172.21.15.79:0/2762395962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9f4004850 0x7ff9f4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0e1af700 1 -- 172.21.15.79:0/2762395962 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f40035b0 con 0x7ff9f4003a00 2023-06-23T18:36:31.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0e9b0700 1 -- 172.21.15.79:0/2762395962 reap_dead start 2023-06-23T18:36:31.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0e1af700 1 --2- 172.21.15.79:0/2762395962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9f4003a00 0x7ff9f4003e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009b60 tx=0x7ff9f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba104d52204a5d65 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:31.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0d1ad700 1 -- 172.21.15.79:0/2762395962 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400e6c0 con 0x7ff9f4003a00 2023-06-23T18:36:31.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0d1ad700 1 -- 172.21.15.79:0/2762395962 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0400e820 con 0x7ff9f4003a00 2023-06-23T18:36:31.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.012+0000 7ffa0d1ad700 1 -- 172.21.15.79:0/2762395962 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400eb90 con 0x7ff9f4003a00 2023-06-23T18:36:31.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.013+0000 7ffa10413700 1 -- 172.21.15.79:0/2762395962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9f4003a00 msgr2=0x7ff9f4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.013+0000 7ffa10413700 1 --2- 172.21.15.79:0/2762395962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9f4003a00 0x7ff9f4003e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009b60 tx=0x7ff9f4005880 comp rx=0 tx=0).stop 2023-06-23T18:36:31.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.013+0000 7ffa10413700 1 -- 172.21.15.79:0/2762395962 shutdown_connections 2023-06-23T18:36:31.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.013+0000 7ffa10413700 1 --2- 172.21.15.79:0/2762395962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9f4003a00 0x7ff9f4003e10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.013+0000 7ffa10413700 1 -- 172.21.15.79:0/2762395962 >> 172.21.15.79:0/2762395962 conn(0x7ffa0806d660 msgr2=0x7ffa0806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:31.024 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.013+0000 7ffa10413700 1 -- 172.21.15.79:0/2762395962 shutdown_connections 2023-06-23T18:36:31.024 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.013+0000 7ffa10413700 1 -- 172.21.15.79:0/2762395962 wait complete. 2023-06-23T18:36:31.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.014+0000 7ffa10413700 1 Processor -- start 2023-06-23T18:36:31.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.014+0000 7ffa10413700 1 -- start start 2023-06-23T18:36:31.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa10413700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9f4003a00 0x7ffa081f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa10413700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa080b04b0 0x7ffa080b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa10413700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa08142bb0 0x7ffa08143020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa10413700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa081f6040 con 0x7ffa08142bb0 2023-06-23T18:36:31.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa10413700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080b0e60 con 0x7ffa080b04b0 2023-06-23T18:36:31.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa10413700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080ae7a0 con 0x7ff9f4003a00 2023-06-23T18:36:31.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa0e9b0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa08142bb0 0x7ffa08143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa0e9b0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa08142bb0 0x7ffa08143020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40472/0 (socket says 172.21.15.79:40472) 2023-06-23T18:36:31.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa0e9b0700 1 -- 172.21.15.79:0/2976537148 learned_addr learned my addr 172.21.15.79:0/2976537148 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:31.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa0d9ae700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa080b04b0 0x7ffa080b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa0e1af700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9f4003a00 0x7ffa081f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa0e9b0700 1 -- 172.21.15.79:0/2976537148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9f4003a00 msgr2=0x7ffa081f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa0e9b0700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9f4003a00 0x7ffa081f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa0e9b0700 1 -- 172.21.15.79:0/2976537148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa080b04b0 msgr2=0x7ffa080b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa0e9b0700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa080b04b0 0x7ffa080b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa0e9b0700 1 -- 172.21.15.79:0/2976537148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa080ae4c0 con 0x7ffa08142bb0 2023-06-23T18:36:31.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.015+0000 7ffa0e1af700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9f4003a00 0x7ffa081f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:31.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.016+0000 7ffa0e9b0700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa08142bb0 0x7ffa08143020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffa08070aa0 tx=0x7ffa0000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:31.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.016+0000 7ff9feffd700 1 -- 172.21.15.79:0/2976537148 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa00002090 con 0x7ffa08142bb0 2023-06-23T18:36:31.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.016+0000 7ff9feffd700 1 -- 172.21.15.79:0/2976537148 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa00003c10 con 0x7ffa08142bb0 2023-06-23T18:36:31.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.016+0000 7ffa10413700 1 -- 172.21.15.79:0/2976537148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa081f5980 con 0x7ffa08142bb0 2023-06-23T18:36:31.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.016+0000 7ff9feffd700 1 -- 172.21.15.79:0/2976537148 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0000d990 con 0x7ffa08142bb0 2023-06-23T18:36:31.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.016+0000 7ffa10413700 1 -- 172.21.15.79:0/2976537148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa081541a0 con 0x7ffa08142bb0 2023-06-23T18:36:31.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.017+0000 7ffa10413700 1 -- 172.21.15.79:0/2976537148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa081f6180 con 0x7ffa08142bb0 2023-06-23T18:36:31.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.017+0000 7ff9feffd700 1 -- 172.21.15.79:0/2976537148 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffa0000daf0 con 0x7ffa08142bb0 2023-06-23T18:36:31.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.017+0000 7ff9feffd700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff9ec06ace0 0x7ff9ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.018+0000 7ff9feffd700 1 -- 172.21.15.79:0/2976537148 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffa00089600 con 0x7ffa08142bb0 2023-06-23T18:36:31.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.018+0000 7ffa0e1af700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff9ec06ace0 0x7ff9ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.018+0000 7ffa0e1af700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff9ec06ace0 0x7ff9ec06d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffa040090b0 tx=0x7ffa04008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:31.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.022+0000 7ff9feffd700 1 -- 172.21.15.79:0/2976537148 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ffa081f6180 con 0x7ffa08142bb0 2023-06-23T18:36:31.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.211+0000 7ffa10413700 1 -- 172.21.15.79:0/2976537148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa081f6180 con 0x7ffa08142bb0 2023-06-23T18:36:31.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.212+0000 7ff9feffd700 1 -- 172.21.15.79:0/2976537148 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7ffa081f6180 con 0x7ffa08142bb0 2023-06-23T18:36:31.217 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:31.218 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:31.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.214+0000 7ffa10413700 1 -- 172.21.15.79:0/2976537148 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff9ec06ace0 msgr2=0x7ff9ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.214+0000 7ffa10413700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff9ec06ace0 0x7ff9ec06d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffa040090b0 tx=0x7ffa04008030 comp rx=0 tx=0).stop 2023-06-23T18:36:31.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.214+0000 7ffa10413700 1 -- 172.21.15.79:0/2976537148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa08142bb0 msgr2=0x7ffa08143020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.214+0000 7ffa10413700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa08142bb0 0x7ffa08143020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffa08070aa0 tx=0x7ffa0000d430 comp rx=0 tx=0).stop 2023-06-23T18:36:31.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.214+0000 7ffa10413700 1 -- 172.21.15.79:0/2976537148 shutdown_connections 2023-06-23T18:36:31.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.214+0000 7ffa10413700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa08142bb0 0x7ffa08143020 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.214+0000 7ffa10413700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9f4003a00 0x7ffa081f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.214+0000 7ffa10413700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff9ec06ace0 0x7ff9ec06d190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.214+0000 7ffa10413700 1 --2- 172.21.15.79:0/2976537148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa080b04b0 0x7ffa080b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.214+0000 7ffa10413700 1 -- 172.21.15.79:0/2976537148 >> 172.21.15.79:0/2976537148 conn(0x7ffa0806d660 msgr2=0x7ffa08069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:31.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.214+0000 7ffa10413700 1 -- 172.21.15.79:0/2976537148 shutdown_connections 2023-06-23T18:36:31.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.214+0000 7ffa10413700 1 -- 172.21.15.79:0/2976537148 wait complete. 2023-06-23T18:36:31.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:31.233 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:31.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.352+0000 7f7d8ffb5700 1 Processor -- start 2023-06-23T18:36:31.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.352+0000 7f7d8ffb5700 1 -- start start 2023-06-23T18:36:31.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.353+0000 7f7d8ffb5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d880cb890 0x7f7d880cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.353+0000 7f7d8ffb5700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d880cc9f0 con 0x7f7d880cc060 2023-06-23T18:36:31.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.353+0000 7f7d8ffb5700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8815a560 con 0x7f7d88156ae0 2023-06-23T18:36:31.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.353+0000 7f7d8ffb5700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880cbd10 con 0x7f7d880cb890 2023-06-23T18:36:31.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.353+0000 7f7d8d550700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f7d880cc060 0x7f7d88071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43518/0 (socket says 172.21.15.79:43518) 2023-06-23T18:36:31.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.353+0000 7f7d8d550700 1 -- 172.21.15.79:0/2890200061 learned_addr learned my addr 172.21.15.79:0/2890200061 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:31.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.353+0000 7f7d8cd4f700 1 -- 172.21.15.79:0/2890200061 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205639952 0 0) 0x7f7d84001110 con 0x7f7d880cc060 2023-06-23T18:36:31.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 -- 172.21.15.79:0/2890200061 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d740035b0 con 0x7f7d880cc060 2023-06-23T18:36:31.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8e552700 1 --2- 172.21.15.79:0/2890200061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d880cb890 0x7f7d880cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 -- 172.21.15.79:0/2890200061 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2231692908 0 0) 0x7f7d740035b0 con 0x7f7d880cc060 2023-06-23T18:36:31.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 -- 172.21.15.79:0/2890200061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d880cb890 msgr2=0x7f7d880cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:31.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 --2- 172.21.15.79:0/2890200061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d880cb890 0x7f7d880cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 -- 172.21.15.79:0/2890200061 >> v1:172.21.15.155:6789/0 conn(0x7f7d88156ae0 legacy=0x7f7d8815d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 -- 172.21.15.79:0/2890200061 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d88159950 con 0x7f7d880cc060 2023-06-23T18:36:31.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 -- 172.21.15.79:0/2890200061 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f7d84001800 con 0x7f7d880cc060 2023-06-23T18:36:31.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 -- 172.21.15.79:0/2890200061 >> v1:172.21.15.79:6789/0 conn(0x7f7d880cc060 legacy=0x7f7d88071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 --2- 172.21.15.79:0/2890200061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d74003a30 0x7f7d74003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 --2- 172.21.15.79:0/2890200061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d74004850 0x7f7d74004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 --2- 172.21.15.79:0/2890200061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d880cb890 0x7f7d74005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 -- 172.21.15.79:0/2890200061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880cbd10 con 0x7f7d74003a30 2023-06-23T18:36:31.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 -- 172.21.15.79:0/2890200061 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7400bac0 con 0x7f7d74004850 2023-06-23T18:36:31.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 -- 172.21.15.79:0/2890200061 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7400bc50 con 0x7f7d880cb890 2023-06-23T18:36:31.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8dd51700 1 --2- 172.21.15.79:0/2890200061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d74003a30 0x7f7d74003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8e552700 1 --2- 172.21.15.79:0/2890200061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d880cb890 0x7f7d74005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8d550700 1 --2- 172.21.15.79:0/2890200061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d74004850 0x7f7d74004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8dd51700 1 -- 172.21.15.79:0/2890200061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d880cb890 msgr2=0x7f7d74005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8dd51700 1 --2- 172.21.15.79:0/2890200061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d880cb890 0x7f7d74005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8dd51700 1 -- 172.21.15.79:0/2890200061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d74004850 msgr2=0x7f7d74004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8dd51700 1 --2- 172.21.15.79:0/2890200061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d74004850 0x7f7d74004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8dd51700 1 -- 172.21.15.79:0/2890200061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d740035b0 con 0x7f7d74003a30 2023-06-23T18:36:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8d550700 1 --2- 172.21.15.79:0/2890200061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d74004850 0x7f7d74004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8dd51700 1 --2- 172.21.15.79:0/2890200061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d74003a30 0x7f7d74003e40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7d780017c0 tx=0x7f7d74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860c668d98126272 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:31.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.354+0000 7f7d8cd4f700 1 -- 172.21.15.79:0/2890200061 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780052d0 con 0x7f7d74003a30 2023-06-23T18:36:31.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.355+0000 7f7d8cd4f700 1 -- 172.21.15.79:0/2890200061 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d78005430 con 0x7f7d74003a30 2023-06-23T18:36:31.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.355+0000 7f7d8cd4f700 1 -- 172.21.15.79:0/2890200061 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780038c0 con 0x7f7d74003a30 2023-06-23T18:36:31.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.355+0000 7f7d8ffb5700 1 -- 172.21.15.79:0/2890200061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d74003a30 msgr2=0x7f7d74003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.355+0000 7f7d8ffb5700 1 --2- 172.21.15.79:0/2890200061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d74003a30 0x7f7d74003e40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7d780017c0 tx=0x7f7d74005880 comp rx=0 tx=0).stop 2023-06-23T18:36:31.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.355+0000 7f7d8e552700 1 -- 172.21.15.79:0/2890200061 reap_dead start 2023-06-23T18:36:31.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.355+0000 7f7d8ffb5700 1 -- 172.21.15.79:0/2890200061 shutdown_connections 2023-06-23T18:36:31.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.355+0000 7f7d8ffb5700 1 -- 172.21.15.79:0/2890200061 >> 172.21.15.79:0/2890200061 conn(0x7f7d8806afa0 msgr2=0x7f7d8806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:31.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.355+0000 7f7d8ffb5700 1 -- 172.21.15.79:0/2890200061 shutdown_connections 2023-06-23T18:36:31.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.355+0000 7f7d8ffb5700 1 -- 172.21.15.79:0/2890200061 wait complete. 2023-06-23T18:36:31.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.357+0000 7f7d8ffb5700 1 Processor -- start 2023-06-23T18:36:31.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.357+0000 7f7d8ffb5700 1 -- start start 2023-06-23T18:36:31.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8ffb5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d74003a30 0x7f7d881440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8ffb5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d881580c0 0x7f7d881efa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8ffb5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d880b26a0 0x7f7d880b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8ffb5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880b3050 con 0x7f7d880b26a0 2023-06-23T18:36:31.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8ffb5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d881470b0 con 0x7f7d881580c0 2023-06-23T18:36:31.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8ffb5700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d881464a0 con 0x7f7d74003a30 2023-06-23T18:36:31.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8e552700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d880b26a0 0x7f7d880b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8e552700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d880b26a0 0x7f7d880b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40492/0 (socket says 172.21.15.79:40492) 2023-06-23T18:36:31.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8e552700 1 -- 172.21.15.79:0/2122467919 learned_addr learned my addr 172.21.15.79:0/2122467919 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:31.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8d550700 1 --2- 172.21.15.79:0/2122467919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d881580c0 0x7f7d881efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8dd51700 1 --2- 172.21.15.79:0/2122467919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d74003a30 0x7f7d881440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8e552700 1 -- 172.21.15.79:0/2122467919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d74003a30 msgr2=0x7f7d881440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8e552700 1 --2- 172.21.15.79:0/2122467919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d74003a30 0x7f7d881440e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8e552700 1 -- 172.21.15.79:0/2122467919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d881580c0 msgr2=0x7f7d881efa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8e552700 1 --2- 172.21.15.79:0/2122467919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d881580c0 0x7f7d881efa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8e552700 1 -- 172.21.15.79:0/2122467919 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d88146780 con 0x7f7d880b26a0 2023-06-23T18:36:31.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8e552700 1 --2- 172.21.15.79:0/2122467919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d880b26a0 0x7f7d880b2b10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7d880b18b0 tx=0x7f7d80011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:31.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d7e7fc700 1 -- 172.21.15.79:0/2122467919 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000e040 con 0x7f7d880b26a0 2023-06-23T18:36:31.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d7e7fc700 1 -- 172.21.15.79:0/2122467919 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8000f040 con 0x7f7d880b26a0 2023-06-23T18:36:31.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8ffb5700 1 -- 172.21.15.79:0/2122467919 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d88146a60 con 0x7f7d880b26a0 2023-06-23T18:36:31.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d7e7fc700 1 -- 172.21.15.79:0/2122467919 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80015430 con 0x7f7d880b26a0 2023-06-23T18:36:31.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.358+0000 7f7d8ffb5700 1 -- 172.21.15.79:0/2122467919 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d88158530 con 0x7f7d880b26a0 2023-06-23T18:36:31.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.359+0000 7f7d6bfff700 1 -- 172.21.15.79:0/2122467919 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d640052f0 con 0x7f7d880b26a0 2023-06-23T18:36:31.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.360+0000 7f7d7e7fc700 1 -- 172.21.15.79:0/2122467919 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d80015590 con 0x7f7d880b26a0 2023-06-23T18:36:31.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.360+0000 7f7d7e7fc700 1 --2- 172.21.15.79:0/2122467919 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7d6c06ace0 0x7f7d6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.360+0000 7f7d7e7fc700 1 -- 172.21.15.79:0/2122467919 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d80089940 con 0x7f7d880b26a0 2023-06-23T18:36:31.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.360+0000 7f7d8dd51700 1 --2- 172.21.15.79:0/2122467919 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7d6c06ace0 0x7f7d6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.360+0000 7f7d8dd51700 1 --2- 172.21.15.79:0/2122467919 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7d6c06ace0 0x7f7d6c06d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7d78015040 tx=0x7f7d78009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:31.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.364+0000 7f7d7e7fc700 1 -- 172.21.15.79:0/2122467919 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7d800549f0 con 0x7f7d880b26a0 2023-06-23T18:36:31.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.549+0000 7f7d6bfff700 1 -- 172.21.15.79:0/2122467919 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7d64005b20 con 0x7f7d880b26a0 2023-06-23T18:36:31.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.549+0000 7f7d7e7fc700 1 -- 172.21.15.79:0/2122467919 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7d80058010 con 0x7f7d880b26a0 2023-06-23T18:36:31.555 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:31.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.551+0000 7f7d6bfff700 1 -- 172.21.15.79:0/2122467919 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7d6c06ace0 msgr2=0x7f7d6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.551+0000 7f7d6bfff700 1 --2- 172.21.15.79:0/2122467919 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7d6c06ace0 0x7f7d6c06d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7d78015040 tx=0x7f7d78009030 comp rx=0 tx=0).stop 2023-06-23T18:36:31.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.551+0000 7f7d6bfff700 1 -- 172.21.15.79:0/2122467919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d880b26a0 msgr2=0x7f7d880b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.551+0000 7f7d6bfff700 1 --2- 172.21.15.79:0/2122467919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d880b26a0 0x7f7d880b2b10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7d880b18b0 tx=0x7f7d80011f80 comp rx=0 tx=0).stop 2023-06-23T18:36:31.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.551+0000 7f7d6bfff700 1 -- 172.21.15.79:0/2122467919 shutdown_connections 2023-06-23T18:36:31.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.551+0000 7f7d6bfff700 1 --2- 172.21.15.79:0/2122467919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d880b26a0 0x7f7d880b2b10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.551+0000 7f7d6bfff700 1 --2- 172.21.15.79:0/2122467919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d74003a30 0x7f7d881440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.551+0000 7f7d6bfff700 1 --2- 172.21.15.79:0/2122467919 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7d6c06ace0 0x7f7d6c06d190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.551+0000 7f7d6bfff700 1 --2- 172.21.15.79:0/2122467919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d881580c0 0x7f7d881efa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.551+0000 7f7d6bfff700 1 -- 172.21.15.79:0/2122467919 >> 172.21.15.79:0/2122467919 conn(0x7f7d8806afa0 msgr2=0x7f7d88152c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:31.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.551+0000 7f7d6bfff700 1 -- 172.21.15.79:0/2122467919 shutdown_connections 2023-06-23T18:36:31.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.551+0000 7f7d6bfff700 1 -- 172.21.15.79:0/2122467919 wait complete. 2023-06-23T18:36:31.568 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:31.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.686+0000 7ff72c38c700 1 Processor -- start 2023-06-23T18:36:31.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.686+0000 7ff72c38c700 1 -- start start 2023-06-23T18:36:31.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.686+0000 7ff72c38c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff7241569c0 0x7ff7241608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.686+0000 7ff72c38c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff7240cd440 0x7ff724069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff72c38c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7240cdc90 con 0x7ff72415d580 2023-06-23T18:36:31.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff72c38c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7240cf570 con 0x7ff7240cd440 2023-06-23T18:36:31.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff72c38c700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72415c8b0 con 0x7ff7241569c0 2023-06-23T18:36:31.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff729927700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff7240cd440 0x7ff724069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff729927700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff7240cd440 0x7ff724069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40496/0 (socket says 172.21.15.79:40496) 2023-06-23T18:36:31.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff729927700 1 -- 172.21.15.79:0/700223828 learned_addr learned my addr 172.21.15.79:0/700223828 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:31.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff72a128700 1 --2- 172.21.15.79:0/700223828 >> v2:172.21.15.155:3300/0 conn(0x7ff7241569c0 0x7ff7241608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff729927700 1 -- 172.21.15.79:0/700223828 >> v2:172.21.15.155:3300/0 conn(0x7ff7241569c0 msgr2=0x7ff7241608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:31.696 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff729927700 1 --2- 172.21.15.79:0/700223828 >> v2:172.21.15.155:3300/0 conn(0x7ff7241569c0 0x7ff7241608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.696 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff729927700 1 -- 172.21.15.79:0/700223828 >> v1:172.21.15.155:6789/0 conn(0x7ff72415d580 legacy=0x7ff7240b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:31.696 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff729927700 1 -- 172.21.15.79:0/700223828 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72415d0f0 con 0x7ff7240cd440 2023-06-23T18:36:31.696 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff729927700 1 --2- 172.21.15.79:0/700223828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7240cd440 0x7ff724069e90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff724070f20 tx=0x7ff720009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2667c2fa32f8c20 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:31.696 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff729126700 1 -- 172.21.15.79:0/700223828 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720004230 con 0x7ff7240cd440 2023-06-23T18:36:31.696 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff729126700 1 -- 172.21.15.79:0/700223828 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff720004390 con 0x7ff7240cd440 2023-06-23T18:36:31.696 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.687+0000 7ff729126700 1 -- 172.21.15.79:0/700223828 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7200036a0 con 0x7ff7240cd440 2023-06-23T18:36:31.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.688+0000 7ff72c38c700 1 -- 172.21.15.79:0/700223828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7240cd440 msgr2=0x7ff724069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.688+0000 7ff72c38c700 1 --2- 172.21.15.79:0/700223828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7240cd440 0x7ff724069e90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff724070f20 tx=0x7ff720009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.688+0000 7ff72c38c700 1 -- 172.21.15.79:0/700223828 shutdown_connections 2023-06-23T18:36:31.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.688+0000 7ff72c38c700 1 --2- 172.21.15.79:0/700223828 >> v2:172.21.15.155:3300/0 conn(0x7ff7241569c0 0x7ff7241608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.688+0000 7ff72c38c700 1 --2- 172.21.15.79:0/700223828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7240cd440 0x7ff724069e90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.688+0000 7ff72c38c700 1 -- 172.21.15.79:0/700223828 >> 172.21.15.79:0/700223828 conn(0x7ff72406afa0 msgr2=0x7ff72406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:31.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.688+0000 7ff72c38c700 1 -- 172.21.15.79:0/700223828 shutdown_connections 2023-06-23T18:36:31.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.688+0000 7ff72c38c700 1 -- 172.21.15.79:0/700223828 wait complete. 2023-06-23T18:36:31.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.689+0000 7ff72c38c700 1 Processor -- start 2023-06-23T18:36:31.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.689+0000 7ff72c38c700 1 -- start start 2023-06-23T18:36:31.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72c38c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7240cd440 0x7ff7241f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72c38c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7241569c0 0x7ff7240c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72c38c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff72415d580 0x7ff7241f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72c38c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7241f6200 con 0x7ff7240cd440 2023-06-23T18:36:31.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72c38c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7241f3330 con 0x7ff7241569c0 2023-06-23T18:36:31.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72c38c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7240b0b10 con 0x7ff72415d580 2023-06-23T18:36:31.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72a128700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7240cd440 0x7ff7241f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72a128700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7240cd440 0x7ff7241f2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40502/0 (socket says 172.21.15.79:40502) 2023-06-23T18:36:31.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72a128700 1 -- 172.21.15.79:0/2050831053 learned_addr learned my addr 172.21.15.79:0/2050831053 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:31.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff729927700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7241569c0 0x7ff7240c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72a929700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff72415d580 0x7ff7241f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72a128700 1 -- 172.21.15.79:0/2050831053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff72415d580 msgr2=0x7ff7241f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72a128700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff72415d580 0x7ff7241f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72a128700 1 -- 172.21.15.79:0/2050831053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7241569c0 msgr2=0x7ff7240c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72a128700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7241569c0 0x7ff7240c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72a128700 1 -- 172.21.15.79:0/2050831053 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720009550 con 0x7ff7240cd440 2023-06-23T18:36:31.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff729927700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7241569c0 0x7ff7240c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:31.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff72a128700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7240cd440 0x7ff7241f2df0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff71400c7f0 tx=0x7ff71400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:31.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.690+0000 7ff71affd700 1 -- 172.21.15.79:0/2050831053 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff714011800 con 0x7ff7240cd440 2023-06-23T18:36:31.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.691+0000 7ff71affd700 1 -- 172.21.15.79:0/2050831053 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff714011960 con 0x7ff7240cd440 2023-06-23T18:36:31.703 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.691+0000 7ff72c38c700 1 -- 172.21.15.79:0/2050831053 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7241f5f20 con 0x7ff7240cd440 2023-06-23T18:36:31.703 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.691+0000 7ff71affd700 1 -- 172.21.15.79:0/2050831053 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff714011cf0 con 0x7ff7240cd440 2023-06-23T18:36:31.703 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.691+0000 7ff72c38c700 1 -- 172.21.15.79:0/2050831053 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7240b0730 con 0x7ff7240cd440 2023-06-23T18:36:31.703 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.691+0000 7ff72c38c700 1 -- 172.21.15.79:0/2050831053 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7240b02b0 con 0x7ff7240cd440 2023-06-23T18:36:31.703 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.692+0000 7ff71affd700 1 -- 172.21.15.79:0/2050831053 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff714018360 con 0x7ff7240cd440 2023-06-23T18:36:31.703 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.692+0000 7ff71affd700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff70806ace0 0x7ff70806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:31.703 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.692+0000 7ff71affd700 1 -- 172.21.15.79:0/2050831053 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff7140894b0 con 0x7ff7240cd440 2023-06-23T18:36:31.704 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.692+0000 7ff729927700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff70806ace0 0x7ff70806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:31.704 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.693+0000 7ff729927700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff70806ace0 0x7ff70806d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff720009130 tx=0x7ff720008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:31.704 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.697+0000 7ff71affd700 1 -- 172.21.15.79:0/2050831053 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff7240b02b0 con 0x7ff7240cd440 2023-06-23T18:36:31.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.884+0000 7ff72c38c700 1 -- 172.21.15.79:0/2050831053 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7240b02b0 con 0x7ff7240cd440 2023-06-23T18:36:31.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.884+0000 7ff71affd700 1 -- 172.21.15.79:0/2050831053 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7240b02b0 con 0x7ff7240cd440 2023-06-23T18:36:31.891 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:31.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.886+0000 7ff72c38c700 1 -- 172.21.15.79:0/2050831053 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff70806ace0 msgr2=0x7ff70806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.886+0000 7ff72c38c700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff70806ace0 0x7ff70806d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff720009130 tx=0x7ff720008030 comp rx=0 tx=0).stop 2023-06-23T18:36:31.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.886+0000 7ff72c38c700 1 -- 172.21.15.79:0/2050831053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7240cd440 msgr2=0x7ff7241f2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:31.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.886+0000 7ff72c38c700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7240cd440 0x7ff7241f2df0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff71400c7f0 tx=0x7ff71400cbb0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.886+0000 7ff72c38c700 1 -- 172.21.15.79:0/2050831053 shutdown_connections 2023-06-23T18:36:31.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.886+0000 7ff72c38c700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff72415d580 0x7ff7241f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.886+0000 7ff72c38c700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7240cd440 0x7ff7241f2df0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.886+0000 7ff72c38c700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff70806ace0 0x7ff70806d190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.886+0000 7ff72c38c700 1 --2- 172.21.15.79:0/2050831053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7241569c0 0x7ff7240c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:31.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.886+0000 7ff72c38c700 1 -- 172.21.15.79:0/2050831053 >> 172.21.15.79:0/2050831053 conn(0x7ff72406afa0 msgr2=0x7ff724071d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:31.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.886+0000 7ff72c38c700 1 -- 172.21.15.79:0/2050831053 shutdown_connections 2023-06-23T18:36:31.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:31.886+0000 7ff72c38c700 1 -- 172.21.15.79:0/2050831053 wait complete. 2023-06-23T18:36:31.903 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:32.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.020+0000 7f7d26a5a700 1 Processor -- start 2023-06-23T18:36:32.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.021+0000 7f7d26a5a700 1 -- start start 2023-06-23T18:36:32.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.021+0000 7f7d26a5a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7d2015d730 0x7f7d200cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.021+0000 7f7d26a5a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d200d2b10 con 0x7f7d200cee50 2023-06-23T18:36:32.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.021+0000 7f7d26a5a700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d200d0e70 con 0x7f7d2015cc50 2023-06-23T18:36:32.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.021+0000 7f7d26a5a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d200d27e0 con 0x7f7d2015d730 2023-06-23T18:36:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.021+0000 7f7d24ff7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7d2015d730 0x7f7d200cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.021+0000 7f7d1f7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f7d200cee50 0x7f7d2015d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43552/0 (socket says 172.21.15.79:43552) 2023-06-23T18:36:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.021+0000 7f7d1f7fe700 1 -- 172.21.15.79:0/2303022908 learned_addr learned my addr 172.21.15.79:0/2303022908 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.022+0000 7f7d24ff7700 1 -- 172.21.15.79:0/2303022908 >> v1:172.21.15.155:6789/0 conn(0x7f7d2015cc50 legacy=0x7f7d200d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.022+0000 7f7d24ff7700 1 -- 172.21.15.79:0/2303022908 >> v1:172.21.15.79:6789/0 conn(0x7f7d200cee50 legacy=0x7f7d2015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.022+0000 7f7d24ff7700 1 -- 172.21.15.79:0/2303022908 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d200cd310 con 0x7f7d2015d730 2023-06-23T18:36:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.022+0000 7f7d1effd700 1 -- 172.21.15.79:0/2303022908 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185769958 0 0) 0x7f7d200d2b10 con 0x7f7d200cee50 2023-06-23T18:36:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.022+0000 7f7d24ff7700 1 --2- 172.21.15.79:0/2303022908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d2015d730 0x7f7d200cf570 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009d90 tx=0x7f7d1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27d1bd71d6cdc04 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:32.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.022+0000 7f7d1effd700 1 -- 172.21.15.79:0/2303022908 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1800e070 con 0x7f7d2015d730 2023-06-23T18:36:32.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.022+0000 7f7d1effd700 1 -- 172.21.15.79:0/2303022908 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1800bbd0 con 0x7f7d2015d730 2023-06-23T18:36:32.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.022+0000 7f7d1effd700 1 -- 172.21.15.79:0/2303022908 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18012430 con 0x7f7d2015d730 2023-06-23T18:36:32.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.023+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2303022908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d2015d730 msgr2=0x7f7d200cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.023+0000 7f7d26a5a700 1 --2- 172.21.15.79:0/2303022908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d2015d730 0x7f7d200cf570 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009d90 tx=0x7f7d1800b330 comp rx=0 tx=0).stop 2023-06-23T18:36:32.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.023+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2303022908 shutdown_connections 2023-06-23T18:36:32.032 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.023+0000 7f7d26a5a700 1 --2- 172.21.15.79:0/2303022908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d2015d730 0x7f7d200cf570 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.032 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.023+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2303022908 >> 172.21.15.79:0/2303022908 conn(0x7f7d200715c0 msgr2=0x7f7d200719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:32.032 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.023+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2303022908 shutdown_connections 2023-06-23T18:36:32.032 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.023+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2303022908 wait complete. 2023-06-23T18:36:32.032 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.024+0000 7f7d26a5a700 1 Processor -- start 2023-06-23T18:36:32.032 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.024+0000 7f7d26a5a700 1 -- start start 2023-06-23T18:36:32.032 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d26a5a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d200cee50 0x7f7d200b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d26a5a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d2015cc50 0x7f7d200b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d26a5a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d2015d730 0x7f7d201f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d26a5a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d200ccb10 con 0x7f7d2015d730 2023-06-23T18:36:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d26a5a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d200ca130 con 0x7f7d200cee50 2023-06-23T18:36:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d26a5a700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d200c9e00 con 0x7f7d2015cc50 2023-06-23T18:36:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d24ff7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d2015d730 0x7f7d201f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d24ff7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d2015d730 0x7f7d201f14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40518/0 (socket says 172.21.15.79:40518) 2023-06-23T18:36:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d24ff7700 1 -- 172.21.15.79:0/2432336856 learned_addr learned my addr 172.21.15.79:0/2432336856 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:32.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d1ffff700 1 --2- 172.21.15.79:0/2432336856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d200cee50 0x7f7d200b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d1f7fe700 1 --2- 172.21.15.79:0/2432336856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d2015cc50 0x7f7d200b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d24ff7700 1 -- 172.21.15.79:0/2432336856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d2015cc50 msgr2=0x7f7d200b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d24ff7700 1 --2- 172.21.15.79:0/2432336856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d2015cc50 0x7f7d200b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d24ff7700 1 -- 172.21.15.79:0/2432336856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d200cee50 msgr2=0x7f7d200b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d24ff7700 1 --2- 172.21.15.79:0/2432336856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d200cee50 0x7f7d200b7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d24ff7700 1 -- 172.21.15.79:0/2432336856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d18009a70 con 0x7f7d2015d730 2023-06-23T18:36:32.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d24ff7700 1 --2- 172.21.15.79:0/2432336856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d2015d730 0x7f7d201f14f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d18014040 tx=0x7f7d18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:32.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.025+0000 7f7d1cff9700 1 -- 172.21.15.79:0/2432336856 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18012430 con 0x7f7d2015d730 2023-06-23T18:36:32.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.026+0000 7f7d1cff9700 1 -- 172.21.15.79:0/2432336856 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d18020070 con 0x7f7d2015d730 2023-06-23T18:36:32.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.026+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2432336856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d200cc550 con 0x7f7d2015d730 2023-06-23T18:36:32.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.026+0000 7f7d1cff9700 1 -- 172.21.15.79:0/2432336856 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d180098f0 con 0x7f7d2015d730 2023-06-23T18:36:32.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.026+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2432336856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d200b3d20 con 0x7f7d2015d730 2023-06-23T18:36:32.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.027+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2432336856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d200c9c40 con 0x7f7d2015d730 2023-06-23T18:36:32.036 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.027+0000 7f7d1cff9700 1 -- 172.21.15.79:0/2432336856 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d18026400 con 0x7f7d2015d730 2023-06-23T18:36:32.036 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.027+0000 7f7d1cff9700 1 --2- 172.21.15.79:0/2432336856 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7d0006ace0 0x7f7d0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.036 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.027+0000 7f7d1cff9700 1 -- 172.21.15.79:0/2432336856 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d1808b380 con 0x7f7d2015d730 2023-06-23T18:36:32.037 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.027+0000 7f7d1ffff700 1 --2- 172.21.15.79:0/2432336856 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7d0006ace0 0x7f7d0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.037 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.028+0000 7f7d1ffff700 1 --2- 172.21.15.79:0/2432336856 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7d0006ace0 0x7f7d0006d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7d10000f60 tx=0x7f7d10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:32.037 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.032+0000 7f7d1cff9700 1 -- 172.21.15.79:0/2432336856 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7d180564c0 con 0x7f7d2015d730 2023-06-23T18:36:32.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.217+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2432336856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d200ca2a0 con 0x7f7d2015d730 2023-06-23T18:36:32.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.217+0000 7f7d1cff9700 1 -- 172.21.15.79:0/2432336856 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d18059ae0 con 0x7f7d2015d730 2023-06-23T18:36:32.223 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.219+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2432336856 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7d0006ace0 msgr2=0x7f7d0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.219+0000 7f7d26a5a700 1 --2- 172.21.15.79:0/2432336856 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7d0006ace0 0x7f7d0006d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7d10000f60 tx=0x7f7d10009040 comp rx=0 tx=0).stop 2023-06-23T18:36:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.220+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2432336856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d2015d730 msgr2=0x7f7d201f14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.220+0000 7f7d26a5a700 1 --2- 172.21.15.79:0/2432336856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d2015d730 0x7f7d201f14f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d18014040 tx=0x7f7d18004ad0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.220+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2432336856 shutdown_connections 2023-06-23T18:36:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.220+0000 7f7d26a5a700 1 --2- 172.21.15.79:0/2432336856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d2015d730 0x7f7d201f14f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.220+0000 7f7d26a5a700 1 --2- 172.21.15.79:0/2432336856 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7d0006ace0 0x7f7d0006d190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.220+0000 7f7d26a5a700 1 --2- 172.21.15.79:0/2432336856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d200cee50 0x7f7d200b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.220+0000 7f7d26a5a700 1 --2- 172.21.15.79:0/2432336856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d2015cc50 0x7f7d200b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.220+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2432336856 >> 172.21.15.79:0/2432336856 conn(0x7f7d200715c0 msgr2=0x7f7d2015def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:32.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.220+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2432336856 shutdown_connections 2023-06-23T18:36:32.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.220+0000 7f7d26a5a700 1 -- 172.21.15.79:0/2432336856 wait complete. 2023-06-23T18:36:32.237 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:32.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.354+0000 7fc943291700 1 Processor -- start 2023-06-23T18:36:32.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.355+0000 7fc943291700 1 -- start start 2023-06-23T18:36:32.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.355+0000 7fc943291700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc93c0cedf0 0x7fc93c0b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.355+0000 7fc943291700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc93c1438f0 0x7fc93c0b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.355+0000 7fc943291700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc93c0d08c0 0x7fc93c0d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.355+0000 7fc943291700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc93c0d1220 con 0x7fc93c1438f0 2023-06-23T18:36:32.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.355+0000 7fc943291700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc93c0b0b10 con 0x7fc93c0cedf0 2023-06-23T18:36:32.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.355+0000 7fc943291700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c0b0df0 con 0x7fc93c0d08c0 2023-06-23T18:36:32.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.356+0000 7fc94082c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc93c1438f0 0x7fc93c0b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.356+0000 7fc94082c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc93c1438f0 0x7fc93c0b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40524/0 (socket says 172.21.15.79:40524) 2023-06-23T18:36:32.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.356+0000 7fc94082c700 1 -- 172.21.15.79:0/3196928754 learned_addr learned my addr 172.21.15.79:0/3196928754 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:32.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.356+0000 7fc94102d700 1 --2- 172.21.15.79:0/3196928754 >> v2:172.21.15.155:3300/0 conn(0x7fc93c0cedf0 0x7fc93c0b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.356+0000 7fc94182e700 1 --2- 172.21.15.79:0/3196928754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc93c0d08c0 0x7fc93c0d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.356+0000 7fc94082c700 1 -- 172.21.15.79:0/3196928754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc93c0d08c0 msgr2=0x7fc93c0d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:32.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.356+0000 7fc94082c700 1 --2- 172.21.15.79:0/3196928754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc93c0d08c0 0x7fc93c0d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.356+0000 7fc94082c700 1 -- 172.21.15.79:0/3196928754 >> v2:172.21.15.155:3300/0 conn(0x7fc93c0cedf0 msgr2=0x7fc93c0b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:32.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.356+0000 7fc94082c700 1 --2- 172.21.15.79:0/3196928754 >> v2:172.21.15.155:3300/0 conn(0x7fc93c0cedf0 0x7fc93c0b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.356+0000 7fc94082c700 1 -- 172.21.15.79:0/3196928754 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c0b10d0 con 0x7fc93c1438f0 2023-06-23T18:36:32.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.356+0000 7fc94082c700 1 --2- 172.21.15.79:0/3196928754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1438f0 0x7fc93c0b1bb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc93c066900 tx=0x7fc938009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0be4a67b71ab13 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:32.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.356+0000 7fc933fff700 1 -- 172.21.15.79:0/3196928754 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938011830 con 0x7fc93c1438f0 2023-06-23T18:36:32.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.356+0000 7fc933fff700 1 -- 172.21.15.79:0/3196928754 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc938011990 con 0x7fc93c1438f0 2023-06-23T18:36:32.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.356+0000 7fc933fff700 1 -- 172.21.15.79:0/3196928754 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938011d20 con 0x7fc93c1438f0 2023-06-23T18:36:32.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.357+0000 7fc943291700 1 -- 172.21.15.79:0/3196928754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1438f0 msgr2=0x7fc93c0b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.357+0000 7fc943291700 1 --2- 172.21.15.79:0/3196928754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1438f0 0x7fc93c0b1bb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc93c066900 tx=0x7fc938009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.357+0000 7fc943291700 1 -- 172.21.15.79:0/3196928754 shutdown_connections 2023-06-23T18:36:32.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.357+0000 7fc943291700 1 --2- 172.21.15.79:0/3196928754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc93c0d08c0 0x7fc93c0d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.357+0000 7fc943291700 1 --2- 172.21.15.79:0/3196928754 >> v2:172.21.15.155:3300/0 conn(0x7fc93c0cedf0 0x7fc93c0b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.357+0000 7fc943291700 1 --2- 172.21.15.79:0/3196928754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1438f0 0x7fc93c0b1bb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.357+0000 7fc943291700 1 -- 172.21.15.79:0/3196928754 >> 172.21.15.79:0/3196928754 conn(0x7fc93c06a7b0 msgr2=0x7fc93c099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:32.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.357+0000 7fc943291700 1 -- 172.21.15.79:0/3196928754 shutdown_connections 2023-06-23T18:36:32.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.357+0000 7fc943291700 1 -- 172.21.15.79:0/3196928754 wait complete. 2023-06-23T18:36:32.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.358+0000 7fc943291700 1 Processor -- start 2023-06-23T18:36:32.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.358+0000 7fc943291700 1 -- start start 2023-06-23T18:36:32.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc943291700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc93c0cedf0 0x7fc93c153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc943291700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc93c0d08c0 0x7fc93c0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc943291700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1438f0 0x7fc93c06b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc943291700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c0cc390 con 0x7fc93c1438f0 2023-06-23T18:36:32.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc943291700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c0cc670 con 0x7fc93c0d08c0 2023-06-23T18:36:32.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc943291700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c0cc950 con 0x7fc93c0cedf0 2023-06-23T18:36:32.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc94182e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1438f0 0x7fc93c06b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc94182e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1438f0 0x7fc93c06b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40534/0 (socket says 172.21.15.79:40534) 2023-06-23T18:36:32.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc94182e700 1 -- 172.21.15.79:0/4008613866 learned_addr learned my addr 172.21.15.79:0/4008613866 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:32.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc94182e700 1 -- 172.21.15.79:0/4008613866 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc93c0cedf0 msgr2=0x7fc93c153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc94182e700 1 --2- 172.21.15.79:0/4008613866 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc93c0cedf0 0x7fc93c153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc94082c700 1 --2- 172.21.15.79:0/4008613866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc93c0d08c0 0x7fc93c0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc94182e700 1 -- 172.21.15.79:0/4008613866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc93c0d08c0 msgr2=0x7fc93c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc94182e700 1 --2- 172.21.15.79:0/4008613866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc93c0d08c0 0x7fc93c0cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc94182e700 1 -- 172.21.15.79:0/4008613866 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc938009550 con 0x7fc93c1438f0 2023-06-23T18:36:32.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc94082c700 1 --2- 172.21.15.79:0/4008613866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc93c0d08c0 0x7fc93c0cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:32.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.359+0000 7fc94182e700 1 --2- 172.21.15.79:0/4008613866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1438f0 0x7fc93c06b990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc93400efc0 tx=0x7fc93400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:32.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.360+0000 7fc931ffb700 1 -- 172.21.15.79:0/4008613866 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93400cd40 con 0x7fc93c1438f0 2023-06-23T18:36:32.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.360+0000 7fc931ffb700 1 -- 172.21.15.79:0/4008613866 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93400f040 con 0x7fc93c1438f0 2023-06-23T18:36:32.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.360+0000 7fc931ffb700 1 -- 172.21.15.79:0/4008613866 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9340146c0 con 0x7fc93c1438f0 2023-06-23T18:36:32.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.360+0000 7fc943291700 1 -- 172.21.15.79:0/4008613866 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93c0ccc30 con 0x7fc93c1438f0 2023-06-23T18:36:32.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.360+0000 7fc943291700 1 -- 172.21.15.79:0/4008613866 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93c153f40 con 0x7fc93c1438f0 2023-06-23T18:36:32.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.361+0000 7fc943291700 1 -- 172.21.15.79:0/4008613866 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc93c0cd150 con 0x7fc93c1438f0 2023-06-23T18:36:32.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.361+0000 7fc931ffb700 1 -- 172.21.15.79:0/4008613866 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc934014820 con 0x7fc93c1438f0 2023-06-23T18:36:32.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.362+0000 7fc931ffb700 1 --2- 172.21.15.79:0/4008613866 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc92006ad10 0x7fc92006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.362+0000 7fc931ffb700 1 -- 172.21.15.79:0/4008613866 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc934089e30 con 0x7fc93c1438f0 2023-06-23T18:36:32.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.362+0000 7fc94102d700 1 --2- 172.21.15.79:0/4008613866 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc92006ad10 0x7fc92006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.362+0000 7fc94102d700 1 --2- 172.21.15.79:0/4008613866 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc92006ad10 0x7fc92006d1c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009ac0 tx=0x7fc92c009580 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:32.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.366+0000 7fc931ffb700 1 -- 172.21.15.79:0/4008613866 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc93c0cd150 con 0x7fc93c1438f0 2023-06-23T18:36:32.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.552+0000 7fc943291700 1 -- 172.21.15.79:0/4008613866 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc93c0cd150 con 0x7fc93c1438f0 2023-06-23T18:36:32.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.552+0000 7fc931ffb700 1 -- 172.21.15.79:0/4008613866 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc934018020 con 0x7fc93c1438f0 2023-06-23T18:36:32.557 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:32.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.554+0000 7fc943291700 1 -- 172.21.15.79:0/4008613866 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc92006ad10 msgr2=0x7fc92006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.554+0000 7fc943291700 1 --2- 172.21.15.79:0/4008613866 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc92006ad10 0x7fc92006d1c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009ac0 tx=0x7fc92c009580 comp rx=0 tx=0).stop 2023-06-23T18:36:32.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.554+0000 7fc943291700 1 -- 172.21.15.79:0/4008613866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1438f0 msgr2=0x7fc93c06b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.554+0000 7fc943291700 1 --2- 172.21.15.79:0/4008613866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1438f0 0x7fc93c06b990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc93400efc0 tx=0x7fc93400c3a0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.554+0000 7fc943291700 1 -- 172.21.15.79:0/4008613866 shutdown_connections 2023-06-23T18:36:32.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.554+0000 7fc943291700 1 --2- 172.21.15.79:0/4008613866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc93c1438f0 0x7fc93c06b990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.554+0000 7fc943291700 1 --2- 172.21.15.79:0/4008613866 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc93c0cedf0 0x7fc93c153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.554+0000 7fc943291700 1 --2- 172.21.15.79:0/4008613866 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc92006ad10 0x7fc92006d1c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.554+0000 7fc943291700 1 --2- 172.21.15.79:0/4008613866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc93c0d08c0 0x7fc93c0cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.554+0000 7fc943291700 1 -- 172.21.15.79:0/4008613866 >> 172.21.15.79:0/4008613866 conn(0x7fc93c06a7b0 msgr2=0x7fc93c06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:32.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.554+0000 7fc943291700 1 -- 172.21.15.79:0/4008613866 shutdown_connections 2023-06-23T18:36:32.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.554+0000 7fc943291700 1 -- 172.21.15.79:0/4008613866 wait complete. 2023-06-23T18:36:32.571 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:36:32.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.687+0000 7f7e71898700 1 Processor -- start 2023-06-23T18:36:32.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.687+0000 7f7e71898700 1 -- start start 2023-06-23T18:36:32.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.687+0000 7f7e71898700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7e6c154910 0x7f7e6c070c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.687+0000 7f7e71898700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6c070230 con 0x7f7e6c0c9410 2023-06-23T18:36:32.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.688+0000 7f7e71898700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6c15b2a0 con 0x7f7e6c15a7e0 2023-06-23T18:36:32.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.688+0000 7f7e71898700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6c15b580 con 0x7f7e6c154910 2023-06-23T18:36:32.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.688+0000 7f7e6affd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7e6c154910 0x7f7e6c070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.688+0000 7f7e6b7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f7e6c0c9410 0x7f7e6c0c3600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43584/0 (socket says 172.21.15.79:43584) 2023-06-23T18:36:32.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.688+0000 7f7e6b7fe700 1 -- 172.21.15.79:0/901221506 learned_addr learned my addr 172.21.15.79:0/901221506 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:32.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.688+0000 7f7e6affd700 1 -- 172.21.15.79:0/901221506 >> v1:172.21.15.155:6789/0 conn(0x7f7e6c15a7e0 legacy=0x7f7e6c0c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:32.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.688+0000 7f7e6affd700 1 -- 172.21.15.79:0/901221506 >> v1:172.21.15.79:6789/0 conn(0x7f7e6c0c9410 legacy=0x7f7e6c0c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.688+0000 7f7e6affd700 1 -- 172.21.15.79:0/901221506 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c070480 con 0x7f7e6c154910 2023-06-23T18:36:32.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.688+0000 7f7e69ffb700 1 -- 172.21.15.79:0/901221506 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916501 0 0) 0x7f7e6c070230 con 0x7f7e6c0c9410 2023-06-23T18:36:32.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.688+0000 7f7e6affd700 1 --2- 172.21.15.79:0/901221506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e6c154910 0x7f7e6c070c80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009840 tx=0x7f7e5c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef68140976707cc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:32.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.688+0000 7f7e69ffb700 1 -- 172.21.15.79:0/901221506 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c004230 con 0x7f7e6c154910 2023-06-23T18:36:32.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.688+0000 7f7e69ffb700 1 -- 172.21.15.79:0/901221506 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c004390 con 0x7f7e6c154910 2023-06-23T18:36:32.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.688+0000 7f7e69ffb700 1 -- 172.21.15.79:0/901221506 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0036a0 con 0x7f7e6c154910 2023-06-23T18:36:32.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.689+0000 7f7e71898700 1 -- 172.21.15.79:0/901221506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e6c154910 msgr2=0x7f7e6c070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.689+0000 7f7e71898700 1 --2- 172.21.15.79:0/901221506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e6c154910 0x7f7e6c070c80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009840 tx=0x7f7e5c009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:32.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.689+0000 7f7e71898700 1 -- 172.21.15.79:0/901221506 shutdown_connections 2023-06-23T18:36:32.698 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.689+0000 7f7e71898700 1 --2- 172.21.15.79:0/901221506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e6c154910 0x7f7e6c070c80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.698 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.689+0000 7f7e71898700 1 -- 172.21.15.79:0/901221506 >> 172.21.15.79:0/901221506 conn(0x7f7e6c0cf3d0 msgr2=0x7f7e6c0cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:32.698 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.689+0000 7f7e71898700 1 -- 172.21.15.79:0/901221506 shutdown_connections 2023-06-23T18:36:32.698 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.689+0000 7f7e71898700 1 -- 172.21.15.79:0/901221506 wait complete. 2023-06-23T18:36:32.698 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.690+0000 7f7e71898700 1 Processor -- start 2023-06-23T18:36:32.698 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.690+0000 7f7e71898700 1 -- start start 2023-06-23T18:36:32.698 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e71898700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e6c0c9410 0x7f7e6c0cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e71898700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c154910 0x7f7e6c0c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e71898700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c15a7e0 0x7f7e6c1590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e71898700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c15e8e0 con 0x7f7e6c0c9410 2023-06-23T18:36:32.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e71898700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c15d9c0 con 0x7f7e6c15a7e0 2023-06-23T18:36:32.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e71898700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c15dc60 con 0x7f7e6c154910 2023-06-23T18:36:32.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e6affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e6c0c9410 0x7f7e6c0cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e6affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e6c0c9410 0x7f7e6c0cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40544/0 (socket says 172.21.15.79:40544) 2023-06-23T18:36:32.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e6affd700 1 -- 172.21.15.79:0/1357631561 learned_addr learned my addr 172.21.15.79:0/1357631561 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:32.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e6b7fe700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c15a7e0 0x7f7e6c1590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e6a7fc700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c154910 0x7f7e6c0c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e6affd700 1 -- 172.21.15.79:0/1357631561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c154910 msgr2=0x7f7e6c0c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e6affd700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c154910 0x7f7e6c0c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e6affd700 1 -- 172.21.15.79:0/1357631561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c15a7e0 msgr2=0x7f7e6c1590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e6affd700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c15a7e0 0x7f7e6c1590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e6affd700 1 -- 172.21.15.79:0/1357631561 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c0094a0 con 0x7f7e6c0c9410 2023-06-23T18:36:32.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e6b7fe700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c15a7e0 0x7f7e6c1590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:32.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e6affd700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e6c0c9410 0x7f7e6c0cc750 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0090b0 tx=0x7f7e5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:32.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.691+0000 7f7e53fff700 1 -- 172.21.15.79:0/1357631561 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c011830 con 0x7f7e6c0c9410 2023-06-23T18:36:32.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.692+0000 7f7e53fff700 1 -- 172.21.15.79:0/1357631561 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c01e070 con 0x7f7e6c0c9410 2023-06-23T18:36:32.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.692+0000 7f7e71898700 1 -- 172.21.15.79:0/1357631561 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6c15df40 con 0x7f7e6c0c9410 2023-06-23T18:36:32.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.692+0000 7f7e53fff700 1 -- 172.21.15.79:0/1357631561 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c011bb0 con 0x7f7e6c0c9410 2023-06-23T18:36:32.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.692+0000 7f7e71898700 1 -- 172.21.15.79:0/1357631561 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6c15e500 con 0x7f7e6c0c9410 2023-06-23T18:36:32.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.692+0000 7f7e71898700 1 -- 172.21.15.79:0/1357631561 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6c0cc530 con 0x7f7e6c0c9410 2023-06-23T18:36:32.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.693+0000 7f7e53fff700 1 -- 172.21.15.79:0/1357631561 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e5c011d10 con 0x7f7e6c0c9410 2023-06-23T18:36:32.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.693+0000 7f7e53fff700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e5806ace0 0x7f7e5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:32.703 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.694+0000 7f7e53fff700 1 -- 172.21.15.79:0/1357631561 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7e5c08a950 con 0x7f7e6c0c9410 2023-06-23T18:36:32.703 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.694+0000 7f7e6a7fc700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e5806ace0 0x7f7e5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:32.703 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.694+0000 7f7e6a7fc700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e5806ace0 0x7f7e5806d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0b0c40 tx=0x7f7e60008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:32.703 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.698+0000 7f7e53fff700 1 -- 172.21.15.79:0/1357631561 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7e5c055970 con 0x7f7e6c0c9410 2023-06-23T18:36:32.907 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.903+0000 7f7e71898700 1 -- 172.21.15.79:0/1357631561 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7e6c0cbbe0 con 0x7f7e6c0c9410 2023-06-23T18:36:32.908 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.903+0000 7f7e53fff700 1 -- 172.21.15.79:0/1357631561 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7e5c058f90 con 0x7f7e6c0c9410 2023-06-23T18:36:32.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.905+0000 7f7e71898700 1 -- 172.21.15.79:0/1357631561 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e5806ace0 msgr2=0x7f7e5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.905+0000 7f7e71898700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e5806ace0 0x7f7e5806d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0b0c40 tx=0x7f7e60008040 comp rx=0 tx=0).stop 2023-06-23T18:36:32.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.905+0000 7f7e71898700 1 -- 172.21.15.79:0/1357631561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e6c0c9410 msgr2=0x7f7e6c0cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:32.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.905+0000 7f7e71898700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e6c0c9410 0x7f7e6c0cc750 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0090b0 tx=0x7f7e5c004750 comp rx=0 tx=0).stop 2023-06-23T18:36:32.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.905+0000 7f7e71898700 1 -- 172.21.15.79:0/1357631561 shutdown_connections 2023-06-23T18:36:32.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.905+0000 7f7e71898700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e5806ace0 0x7f7e5806d190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.905+0000 7f7e71898700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c15a7e0 0x7f7e6c1590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.905+0000 7f7e71898700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e6c0c9410 0x7f7e6c0cc750 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.905+0000 7f7e71898700 1 --2- 172.21.15.79:0/1357631561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c154910 0x7f7e6c0c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:32.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.905+0000 7f7e71898700 1 -- 172.21.15.79:0/1357631561 >> 172.21.15.79:0/1357631561 conn(0x7f7e6c0cf3d0 msgr2=0x7f7e6c0b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:32.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.905+0000 7f7e71898700 1 -- 172.21.15.79:0/1357631561 shutdown_connections 2023-06-23T18:36:32.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:32.905+0000 7f7e71898700 1 -- 172.21.15.79:0/1357631561 wait complete. 2023-06-23T18:36:32.912 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-23T18:36:32.922 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:36:32.922 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:36:32.943 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:33.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.060+0000 7f4a6e0e5700 1 Processor -- start 2023-06-23T18:36:33.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.060+0000 7f4a6e0e5700 1 -- start start 2023-06-23T18:36:33.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.061+0000 7f4a6e0e5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a680b4f20 0x7f4a680c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.061+0000 7f4a6e0e5700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4a680b1f40 0x7f4a68069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.061+0000 7f4a6e0e5700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a6815a810 con 0x7f4a6806a3d0 2023-06-23T18:36:33.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.061+0000 7f4a6e0e5700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a6806a850 con 0x7f4a680b1f40 2023-06-23T18:36:33.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.061+0000 7f4a6e0e5700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a680b52f0 con 0x7f4a680b4f20 2023-06-23T18:36:33.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.061+0000 7f4a67fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f4a6806a3d0 0x7f4a6815df00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43594/0 (socket says 172.21.15.79:43594) 2023-06-23T18:36:33.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.061+0000 7f4a67fff700 1 -- 172.21.15.79:0/2215760952 learned_addr learned my addr 172.21.15.79:0/2215760952 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:33.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.061+0000 7f4a667fc700 1 -- 172.21.15.79:0/2215760952 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389303773 0 0) 0x7f4a6815a810 con 0x7f4a6806a3d0 2023-06-23T18:36:33.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.061+0000 7f4a667fc700 1 -- 172.21.15.79:0/2215760952 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a540035b0 con 0x7f4a6806a3d0 2023-06-23T18:36:33.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.061+0000 7f4a66ffd700 1 --2- 172.21.15.79:0/2215760952 >> v2:172.21.15.155:3300/0 conn(0x7f4a680b1f40 0x7f4a68069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.061+0000 7f4a677fe700 1 --2- 172.21.15.79:0/2215760952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a680b4f20 0x7f4a680c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 -- 172.21.15.79:0/2215760952 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 300934535 0 0) 0x7f4a540035b0 con 0x7f4a6806a3d0 2023-06-23T18:36:33.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 -- 172.21.15.79:0/2215760952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a680b4f20 msgr2=0x7f4a680c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:33.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 --2- 172.21.15.79:0/2215760952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a680b4f20 0x7f4a680c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 -- 172.21.15.79:0/2215760952 >> v2:172.21.15.155:3300/0 conn(0x7f4a680b1f40 msgr2=0x7f4a68069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:33.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 --2- 172.21.15.79:0/2215760952 >> v2:172.21.15.155:3300/0 conn(0x7f4a680b1f40 0x7f4a68069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 -- 172.21.15.79:0/2215760952 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a68071530 con 0x7f4a6806a3d0 2023-06-23T18:36:33.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 -- 172.21.15.79:0/2215760952 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f4a600019f0 con 0x7f4a6806a3d0 2023-06-23T18:36:33.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 -- 172.21.15.79:0/2215760952 >> v1:172.21.15.79:6789/0 conn(0x7f4a6806a3d0 legacy=0x7f4a6815df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 --2- 172.21.15.79:0/2215760952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a54003a00 0x7f4a54003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 --2- 172.21.15.79:0/2215760952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a54004430 0x7f4a540048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 --2- 172.21.15.79:0/2215760952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a680b4f20 0x7f4a54004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 -- 172.21.15.79:0/2215760952 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6806a850 con 0x7f4a54004430 2023-06-23T18:36:33.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 -- 172.21.15.79:0/2215760952 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a680b52f0 con 0x7f4a54003a00 2023-06-23T18:36:33.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 -- 172.21.15.79:0/2215760952 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5400ba10 con 0x7f4a680b4f20 2023-06-23T18:36:33.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a66ffd700 1 --2- 172.21.15.79:0/2215760952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a54004430 0x7f4a540048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a677fe700 1 --2- 172.21.15.79:0/2215760952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a54003a00 0x7f4a54003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a67fff700 1 --2- 172.21.15.79:0/2215760952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a680b4f20 0x7f4a54004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a66ffd700 1 -- 172.21.15.79:0/2215760952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a680b4f20 msgr2=0x7f4a54004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:33.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a66ffd700 1 --2- 172.21.15.79:0/2215760952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a680b4f20 0x7f4a54004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a66ffd700 1 -- 172.21.15.79:0/2215760952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a54003a00 msgr2=0x7f4a54003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:33.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a66ffd700 1 --2- 172.21.15.79:0/2215760952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a54003a00 0x7f4a54003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a66ffd700 1 -- 172.21.15.79:0/2215760952 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a540035b0 con 0x7f4a54004430 2023-06-23T18:36:33.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a66ffd700 1 --2- 172.21.15.79:0/2215760952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a54004430 0x7f4a540048f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009b60 tx=0x7f4a54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c266730e9fd5d5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:33.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 -- 172.21.15.79:0/2215760952 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5800e6c0 con 0x7f4a54004430 2023-06-23T18:36:33.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 -- 172.21.15.79:0/2215760952 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5800e820 con 0x7f4a54004430 2023-06-23T18:36:33.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.062+0000 7f4a667fc700 1 -- 172.21.15.79:0/2215760952 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5800eb90 con 0x7f4a54004430 2023-06-23T18:36:33.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.063+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/2215760952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a54004430 msgr2=0x7f4a540048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.063+0000 7f4a6e0e5700 1 --2- 172.21.15.79:0/2215760952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a54004430 0x7f4a540048f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009b60 tx=0x7f4a54005880 comp rx=0 tx=0).stop 2023-06-23T18:36:33.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.063+0000 7f4a67fff700 1 -- 172.21.15.79:0/2215760952 reap_dead start 2023-06-23T18:36:33.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.063+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/2215760952 shutdown_connections 2023-06-23T18:36:33.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.063+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/2215760952 >> 172.21.15.79:0/2215760952 conn(0x7f4a6806d660 msgr2=0x7f4a6806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:33.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.063+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/2215760952 shutdown_connections 2023-06-23T18:36:33.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.063+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/2215760952 wait complete. 2023-06-23T18:36:33.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.064+0000 7f4a6e0e5700 1 Processor -- start 2023-06-23T18:36:33.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.064+0000 7f4a6e0e5700 1 -- start start 2023-06-23T18:36:33.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a6e0e5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a54004430 0x7f4a680c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a6e0e5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a68154fb0 0x7f4a68155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a6e0e5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a68156c60 0x7f4a681570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a6e0e5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a681e2450 con 0x7f4a68154fb0 2023-06-23T18:36:33.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a6e0e5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a68155960 con 0x7f4a68156c60 2023-06-23T18:36:33.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a6e0e5700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a681578e0 con 0x7f4a54004430 2023-06-23T18:36:33.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a66ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a68154fb0 0x7f4a68155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a66ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a68154fb0 0x7f4a68155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40564/0 (socket says 172.21.15.79:40564) 2023-06-23T18:36:33.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a66ffd700 1 -- 172.21.15.79:0/4089270956 learned_addr learned my addr 172.21.15.79:0/4089270956 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:33.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a677fe700 1 --2- 172.21.15.79:0/4089270956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a54004430 0x7f4a680c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a67fff700 1 --2- 172.21.15.79:0/4089270956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a68156c60 0x7f4a681570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a66ffd700 1 -- 172.21.15.79:0/4089270956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a54004430 msgr2=0x7f4a680c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a66ffd700 1 --2- 172.21.15.79:0/4089270956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a54004430 0x7f4a680c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a66ffd700 1 -- 172.21.15.79:0/4089270956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a68156c60 msgr2=0x7f4a681570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a66ffd700 1 --2- 172.21.15.79:0/4089270956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a68156c60 0x7f4a681570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.065+0000 7f4a66ffd700 1 -- 172.21.15.79:0/4089270956 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a681e2870 con 0x7f4a68154fb0 2023-06-23T18:36:33.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.066+0000 7f4a66ffd700 1 --2- 172.21.15.79:0/4089270956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a68154fb0 0x7f4a68155420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4a54005760 tx=0x7f4a580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:33.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.066+0000 7f4a5ffff700 1 -- 172.21.15.79:0/4089270956 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5800e6c0 con 0x7f4a68154fb0 2023-06-23T18:36:33.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.066+0000 7f4a5ffff700 1 -- 172.21.15.79:0/4089270956 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a58003ad0 con 0x7f4a68154fb0 2023-06-23T18:36:33.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.066+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/4089270956 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a68157610 con 0x7f4a68154fb0 2023-06-23T18:36:33.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.066+0000 7f4a5ffff700 1 -- 172.21.15.79:0/4089270956 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58003e80 con 0x7f4a68154fb0 2023-06-23T18:36:33.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.066+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/4089270956 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a681e1d90 con 0x7f4a68154fb0 2023-06-23T18:36:33.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.067+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/4089270956 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6806e3d0 con 0x7f4a68154fb0 2023-06-23T18:36:33.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.067+0000 7f4a5ffff700 1 -- 172.21.15.79:0/4089270956 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a58013470 con 0x7f4a68154fb0 2023-06-23T18:36:33.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.067+0000 7f4a5ffff700 1 --2- 172.21.15.79:0/4089270956 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4a4806ad10 0x7f4a4806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.068+0000 7f4a5ffff700 1 -- 172.21.15.79:0/4089270956 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4a5808b120 con 0x7f4a68154fb0 2023-06-23T18:36:33.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.068+0000 7f4a677fe700 1 --2- 172.21.15.79:0/4089270956 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4a4806ad10 0x7f4a4806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.068+0000 7f4a677fe700 1 --2- 172.21.15.79:0/4089270956 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4a4806ad10 0x7f4a4806d1c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009ac0 tx=0x7f4a50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:33.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.072+0000 7f4a5ffff700 1 -- 172.21.15.79:0/4089270956 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4a58056380 con 0x7f4a68154fb0 2023-06-23T18:36:33.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.263+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/4089270956 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a681e0880 con 0x7f4a68154fb0 2023-06-23T18:36:33.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.264+0000 7f4a5ffff700 1 -- 172.21.15.79:0/4089270956 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7f4a580599a0 con 0x7f4a68154fb0 2023-06-23T18:36:33.270 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:33.270 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:33.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.266+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/4089270956 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4a4806ad10 msgr2=0x7f4a4806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.266+0000 7f4a6e0e5700 1 --2- 172.21.15.79:0/4089270956 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4a4806ad10 0x7f4a4806d1c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009ac0 tx=0x7f4a50008040 comp rx=0 tx=0).stop 2023-06-23T18:36:33.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.266+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/4089270956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a68154fb0 msgr2=0x7f4a68155420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.266+0000 7f4a6e0e5700 1 --2- 172.21.15.79:0/4089270956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a68154fb0 0x7f4a68155420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4a54005760 tx=0x7f4a580047c0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.267+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/4089270956 shutdown_connections 2023-06-23T18:36:33.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.267+0000 7f4a6e0e5700 1 --2- 172.21.15.79:0/4089270956 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4a4806ad10 0x7f4a4806d1c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.267+0000 7f4a6e0e5700 1 --2- 172.21.15.79:0/4089270956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a68156c60 0x7f4a681570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.267+0000 7f4a6e0e5700 1 --2- 172.21.15.79:0/4089270956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a54004430 0x7f4a680c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.267+0000 7f4a6e0e5700 1 --2- 172.21.15.79:0/4089270956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a68154fb0 0x7f4a68155420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.267+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/4089270956 >> 172.21.15.79:0/4089270956 conn(0x7f4a6806d660 msgr2=0x7f4a681e1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:33.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.267+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/4089270956 shutdown_connections 2023-06-23T18:36:33.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.267+0000 7f4a6e0e5700 1 -- 172.21.15.79:0/4089270956 wait complete. 2023-06-23T18:36:33.283 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:33.284 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:33.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.401+0000 7f8bd8073700 1 Processor -- start 2023-06-23T18:36:33.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.401+0000 7f8bd8073700 1 -- start start 2023-06-23T18:36:33.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.402+0000 7f8bd8073700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8bd01569c0 0x7f8bd01608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.402+0000 7f8bd8073700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bd00cd440 0x7f8bd0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.402+0000 7f8bd8073700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd00cdc90 con 0x7f8bd015d580 2023-06-23T18:36:33.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.402+0000 7f8bd8073700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd00cf570 con 0x7f8bd01569c0 2023-06-23T18:36:33.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.402+0000 7f8bd8073700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd015c8b0 con 0x7f8bd00cd440 2023-06-23T18:36:33.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.402+0000 7f8bd5e0f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8bd01569c0 0x7f8bd01608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.402+0000 7f8bd6610700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f8bd015d580 0x7f8bd00b12f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43614/0 (socket says 172.21.15.79:43614) 2023-06-23T18:36:33.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.402+0000 7f8bd6610700 1 -- 172.21.15.79:0/1289598586 learned_addr learned my addr 172.21.15.79:0/1289598586 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:33.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.403+0000 7f8bd560e700 1 --2- 172.21.15.79:0/1289598586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bd00cd440 0x7f8bd0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.403+0000 7f8bd5e0f700 1 -- 172.21.15.79:0/1289598586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bd00cd440 msgr2=0x7f8bd0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:33.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.403+0000 7f8bd5e0f700 1 --2- 172.21.15.79:0/1289598586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bd00cd440 0x7f8bd0069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.403+0000 7f8bd5e0f700 1 -- 172.21.15.79:0/1289598586 >> v1:172.21.15.79:6789/0 conn(0x7f8bd015d580 legacy=0x7f8bd00b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.403+0000 7f8bd5e0f700 1 -- 172.21.15.79:0/1289598586 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd015d0f0 con 0x7f8bd01569c0 2023-06-23T18:36:33.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.403+0000 7f8bd4e0d700 1 -- 172.21.15.79:0/1289598586 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270514574 0 0) 0x7f8bc8001490 con 0x7f8bd015d580 2023-06-23T18:36:33.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.403+0000 7f8bd5e0f700 1 --2- 172.21.15.79:0/1289598586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8bd01569c0 0x7f8bd01608b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009840 tx=0x7f8bc0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5efae639d891141 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:33.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.403+0000 7f8bd4e0d700 1 -- 172.21.15.79:0/1289598586 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0011830 con 0x7f8bd01569c0 2023-06-23T18:36:33.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.403+0000 7f8bd4e0d700 1 -- 172.21.15.79:0/1289598586 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc0011990 con 0x7f8bd01569c0 2023-06-23T18:36:33.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.403+0000 7f8bd4e0d700 1 -- 172.21.15.79:0/1289598586 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0011d20 con 0x7f8bd01569c0 2023-06-23T18:36:33.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.403+0000 7f8bd8073700 1 -- 172.21.15.79:0/1289598586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8bd01569c0 msgr2=0x7f8bd01608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.403+0000 7f8bd8073700 1 --2- 172.21.15.79:0/1289598586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8bd01569c0 0x7f8bd01608b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009840 tx=0x7f8bc0009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:33.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.404+0000 7f8bd8073700 1 -- 172.21.15.79:0/1289598586 shutdown_connections 2023-06-23T18:36:33.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.404+0000 7f8bd8073700 1 --2- 172.21.15.79:0/1289598586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8bd01569c0 0x7f8bd01608b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.404+0000 7f8bd8073700 1 --2- 172.21.15.79:0/1289598586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bd00cd440 0x7f8bd0069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.404+0000 7f8bd8073700 1 -- 172.21.15.79:0/1289598586 >> 172.21.15.79:0/1289598586 conn(0x7f8bd006afa0 msgr2=0x7f8bd006b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:33.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.404+0000 7f8bd8073700 1 -- 172.21.15.79:0/1289598586 shutdown_connections 2023-06-23T18:36:33.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.404+0000 7f8bd8073700 1 -- 172.21.15.79:0/1289598586 wait complete. 2023-06-23T18:36:33.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.405+0000 7f8bd8073700 1 Processor -- start 2023-06-23T18:36:33.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.405+0000 7f8bd8073700 1 -- start start 2023-06-23T18:36:33.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.405+0000 7f8bd8073700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8bd00cd440 0x7f8bd0144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.405+0000 7f8bd8073700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd01569c0 0x7f8bd00cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd8073700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bd015d580 0x7f8bd00cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd8073700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd0145a90 con 0x7f8bd00cd440 2023-06-23T18:36:33.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd8073700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd00c6300 con 0x7f8bd01569c0 2023-06-23T18:36:33.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd8073700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd0145450 con 0x7f8bd015d580 2023-06-23T18:36:33.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd5e0f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8bd00cd440 0x7f8bd0144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd5e0f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8bd00cd440 0x7f8bd0144f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40574/0 (socket says 172.21.15.79:40574) 2023-06-23T18:36:33.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd5e0f700 1 -- 172.21.15.79:0/2837220925 learned_addr learned my addr 172.21.15.79:0/2837220925 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:33.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd560e700 1 --2- 172.21.15.79:0/2837220925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd01569c0 0x7f8bd00cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd5e0f700 1 -- 172.21.15.79:0/2837220925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bd015d580 msgr2=0x7f8bd00cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd5e0f700 1 --2- 172.21.15.79:0/2837220925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bd015d580 0x7f8bd00cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd5e0f700 1 -- 172.21.15.79:0/2837220925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd01569c0 msgr2=0x7f8bd00cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd5e0f700 1 --2- 172.21.15.79:0/2837220925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd01569c0 0x7f8bd00cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd5e0f700 1 -- 172.21.15.79:0/2837220925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc00094a0 con 0x7f8bd00cd440 2023-06-23T18:36:33.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd560e700 1 --2- 172.21.15.79:0/2837220925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd01569c0 0x7f8bd00cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:33.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd5e0f700 1 --2- 172.21.15.79:0/2837220925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8bd00cd440 0x7f8bd0144f10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00d0970 tx=0x7f8bc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:33.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bc67fc700 1 -- 172.21.15.79:0/2837220925 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00038a0 con 0x7f8bd00cd440 2023-06-23T18:36:33.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bc67fc700 1 -- 172.21.15.79:0/2837220925 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc001f070 con 0x7f8bd00cd440 2023-06-23T18:36:33.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd8073700 1 -- 172.21.15.79:0/2837220925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd00c9510 con 0x7f8bd00cd440 2023-06-23T18:36:33.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bc67fc700 1 -- 172.21.15.79:0/2837220925 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0003c40 con 0x7f8bd00cd440 2023-06-23T18:36:33.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.406+0000 7f8bd8073700 1 -- 172.21.15.79:0/2837220925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd00c9ad0 con 0x7f8bd00cd440 2023-06-23T18:36:33.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.407+0000 7f8bd8073700 1 -- 172.21.15.79:0/2837220925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd00672c0 con 0x7f8bd00cd440 2023-06-23T18:36:33.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.408+0000 7f8bc67fc700 1 -- 172.21.15.79:0/2837220925 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8bc0026480 con 0x7f8bd00cd440 2023-06-23T18:36:33.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.408+0000 7f8bc67fc700 1 --2- 172.21.15.79:0/2837220925 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8bb406ace0 0x7f8bb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.408+0000 7f8bc67fc700 1 -- 172.21.15.79:0/2837220925 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8bc0021070 con 0x7f8bd00cd440 2023-06-23T18:36:33.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.408+0000 7f8bd560e700 1 --2- 172.21.15.79:0/2837220925 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8bb406ace0 0x7f8bb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.408+0000 7f8bd560e700 1 --2- 172.21.15.79:0/2837220925 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8bb406ace0 0x7f8bb406d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00c5730 tx=0x7f8bcc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:33.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.412+0000 7f8bc67fc700 1 -- 172.21.15.79:0/2837220925 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8bc0055fa0 con 0x7f8bd00cd440 2023-06-23T18:36:33.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.603+0000 7f8bd8073700 1 -- 172.21.15.79:0/2837220925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8bd00ccb70 con 0x7f8bd00cd440 2023-06-23T18:36:33.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.604+0000 7f8bc67fc700 1 -- 172.21.15.79:0/2837220925 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8bc00595c0 con 0x7f8bd00cd440 2023-06-23T18:36:33.609 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:33.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.606+0000 7f8bd8073700 1 -- 172.21.15.79:0/2837220925 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8bb406ace0 msgr2=0x7f8bb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.606+0000 7f8bd8073700 1 --2- 172.21.15.79:0/2837220925 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8bb406ace0 0x7f8bb406d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00c5730 tx=0x7f8bcc008040 comp rx=0 tx=0).stop 2023-06-23T18:36:33.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.606+0000 7f8bd8073700 1 -- 172.21.15.79:0/2837220925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8bd00cd440 msgr2=0x7f8bd0144f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.606+0000 7f8bd8073700 1 --2- 172.21.15.79:0/2837220925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8bd00cd440 0x7f8bd0144f10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00d0970 tx=0x7f8bc0004750 comp rx=0 tx=0).stop 2023-06-23T18:36:33.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.606+0000 7f8bd8073700 1 -- 172.21.15.79:0/2837220925 shutdown_connections 2023-06-23T18:36:33.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.606+0000 7f8bd8073700 1 --2- 172.21.15.79:0/2837220925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bd015d580 0x7f8bd00cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.606+0000 7f8bd8073700 1 --2- 172.21.15.79:0/2837220925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8bd00cd440 0x7f8bd0144f10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.606+0000 7f8bd8073700 1 --2- 172.21.15.79:0/2837220925 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8bb406ace0 0x7f8bb406d190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.606+0000 7f8bd8073700 1 --2- 172.21.15.79:0/2837220925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd01569c0 0x7f8bd00cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.606+0000 7f8bd8073700 1 -- 172.21.15.79:0/2837220925 >> 172.21.15.79:0/2837220925 conn(0x7f8bd006afa0 msgr2=0x7f8bd00b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:33.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.606+0000 7f8bd8073700 1 -- 172.21.15.79:0/2837220925 shutdown_connections 2023-06-23T18:36:33.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.606+0000 7f8bd8073700 1 -- 172.21.15.79:0/2837220925 wait complete. 2023-06-23T18:36:33.623 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:33.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.741+0000 7fef26017700 1 Processor -- start 2023-06-23T18:36:33.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.741+0000 7fef26017700 1 -- start start 2023-06-23T18:36:33.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef26017700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fef20156ae0 0x7fef2015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef26017700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fef200cc4b0 0x7fef20071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef26017700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2015daf0 con 0x7fef200cc060 2023-06-23T18:36:33.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef26017700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2015a560 con 0x7fef20156ae0 2023-06-23T18:36:33.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef26017700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef20159950 con 0x7fef200cc4b0 2023-06-23T18:36:33.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef1f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fef20156ae0 0x7fef2015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef1effd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fef200cc060 0x7fef20071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43626/0 (socket says 172.21.15.79:43626) 2023-06-23T18:36:33.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef1f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fef20156ae0 0x7fef2015d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40582/0 (socket says 172.21.15.79:40582) 2023-06-23T18:36:33.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef1effd700 1 -- 172.21.15.79:0/1941568831 learned_addr learned my addr 172.21.15.79:0/1941568831 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:33.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef1ffff700 1 --2- 172.21.15.79:0/1941568831 >> v2:172.21.15.155:3300/0 conn(0x7fef200cc4b0 0x7fef20071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef1f7fe700 1 -- 172.21.15.79:0/1941568831 >> v2:172.21.15.155:3300/0 conn(0x7fef200cc4b0 msgr2=0x7fef20071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:33.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef1f7fe700 1 --2- 172.21.15.79:0/1941568831 >> v2:172.21.15.155:3300/0 conn(0x7fef200cc4b0 0x7fef20071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef1f7fe700 1 -- 172.21.15.79:0/1941568831 >> v1:172.21.15.79:6789/0 conn(0x7fef200cc060 legacy=0x7fef20071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef1f7fe700 1 -- 172.21.15.79:0/1941568831 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef20159c30 con 0x7fef20156ae0 2023-06-23T18:36:33.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef1e7fc700 1 -- 172.21.15.79:0/1941568831 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648832537 0 0) 0x7fef2015daf0 con 0x7fef200cc060 2023-06-23T18:36:33.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef1ffff700 1 --2- 172.21.15.79:0/1941568831 >> v2:172.21.15.155:3300/0 conn(0x7fef200cc4b0 0x7fef20071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:33.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef1f7fe700 1 --2- 172.21.15.79:0/1941568831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef20156ae0 0x7fef2015d5b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fef200c68d0 tx=0x7fef14009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93fd285d5308a02 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:33.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.742+0000 7fef1e7fc700 1 -- 172.21.15.79:0/1941568831 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14011830 con 0x7fef20156ae0 2023-06-23T18:36:33.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.743+0000 7fef1e7fc700 1 -- 172.21.15.79:0/1941568831 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef14011990 con 0x7fef20156ae0 2023-06-23T18:36:33.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.743+0000 7fef1e7fc700 1 -- 172.21.15.79:0/1941568831 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14011d20 con 0x7fef20156ae0 2023-06-23T18:36:33.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.743+0000 7fef26017700 1 -- 172.21.15.79:0/1941568831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef20156ae0 msgr2=0x7fef2015d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.743+0000 7fef26017700 1 --2- 172.21.15.79:0/1941568831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef20156ae0 0x7fef2015d5b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fef200c68d0 tx=0x7fef14009b20 comp rx=0 tx=0).stop 2023-06-23T18:36:33.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.743+0000 7fef26017700 1 -- 172.21.15.79:0/1941568831 shutdown_connections 2023-06-23T18:36:33.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.743+0000 7fef26017700 1 --2- 172.21.15.79:0/1941568831 >> v2:172.21.15.155:3300/0 conn(0x7fef200cc4b0 0x7fef20071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.743+0000 7fef26017700 1 --2- 172.21.15.79:0/1941568831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef20156ae0 0x7fef2015d5b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.743+0000 7fef26017700 1 -- 172.21.15.79:0/1941568831 >> 172.21.15.79:0/1941568831 conn(0x7fef2006afa0 msgr2=0x7fef2006b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:33.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.743+0000 7fef26017700 1 -- 172.21.15.79:0/1941568831 shutdown_connections 2023-06-23T18:36:33.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.743+0000 7fef26017700 1 -- 172.21.15.79:0/1941568831 wait complete. 2023-06-23T18:36:33.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.744+0000 7fef26017700 1 Processor -- start 2023-06-23T18:36:33.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.745+0000 7fef26017700 1 -- start start 2023-06-23T18:36:33.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.745+0000 7fef26017700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef200cc060 0x7fef20154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.745+0000 7fef26017700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef200cc4b0 0x7fef20153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.745+0000 7fef26017700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef20156ae0 0x7fef200b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.745+0000 7fef26017700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef200c8930 con 0x7fef20156ae0 2023-06-23T18:36:33.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.745+0000 7fef26017700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef200c8650 con 0x7fef200cc4b0 2023-06-23T18:36:33.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.745+0000 7fef26017700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef200b31e0 con 0x7fef200cc060 2023-06-23T18:36:33.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.745+0000 7fef1ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef20156ae0 0x7fef200b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.745+0000 7fef1ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef20156ae0 0x7fef200b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40592/0 (socket says 172.21.15.79:40592) 2023-06-23T18:36:33.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.745+0000 7fef1ffff700 1 -- 172.21.15.79:0/1289302742 learned_addr learned my addr 172.21.15.79:0/1289302742 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:33.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.746+0000 7fef1effd700 1 --2- 172.21.15.79:0/1289302742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef200cc4b0 0x7fef20153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.746+0000 7fef1f7fe700 1 --2- 172.21.15.79:0/1289302742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef200cc060 0x7fef20154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.746+0000 7fef1ffff700 1 -- 172.21.15.79:0/1289302742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef200cc060 msgr2=0x7fef20154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.746+0000 7fef1ffff700 1 --2- 172.21.15.79:0/1289302742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef200cc060 0x7fef20154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.746+0000 7fef1ffff700 1 -- 172.21.15.79:0/1289302742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef200cc4b0 msgr2=0x7fef20153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.746+0000 7fef1ffff700 1 --2- 172.21.15.79:0/1289302742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef200cc4b0 0x7fef20153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.746+0000 7fef1ffff700 1 -- 172.21.15.79:0/1289302742 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef140094a0 con 0x7fef20156ae0 2023-06-23T18:36:33.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.746+0000 7fef1ffff700 1 --2- 172.21.15.79:0/1289302742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef20156ae0 0x7fef200b3cc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fef18007fc0 tx=0x7fef1800e100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:33.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.746+0000 7fef07fff700 1 -- 172.21.15.79:0/1289302742 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18011070 con 0x7fef20156ae0 2023-06-23T18:36:33.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.746+0000 7fef07fff700 1 -- 172.21.15.79:0/1289302742 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1800c2e0 con 0x7fef20156ae0 2023-06-23T18:36:33.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.746+0000 7fef26017700 1 -- 172.21.15.79:0/1289302742 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef200b34c0 con 0x7fef20156ae0 2023-06-23T18:36:33.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.746+0000 7fef07fff700 1 -- 172.21.15.79:0/1289302742 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1800eab0 con 0x7fef20156ae0 2023-06-23T18:36:33.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.746+0000 7fef26017700 1 -- 172.21.15.79:0/1289302742 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef20154e10 con 0x7fef20156ae0 2023-06-23T18:36:33.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.747+0000 7fef26017700 1 -- 172.21.15.79:0/1289302742 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef200b39e0 con 0x7fef20156ae0 2023-06-23T18:36:33.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.747+0000 7fef07fff700 1 -- 172.21.15.79:0/1289302742 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef1800ec10 con 0x7fef20156ae0 2023-06-23T18:36:33.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.748+0000 7fef07fff700 1 --2- 172.21.15.79:0/1289302742 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fef0c06ace0 0x7fef0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:33.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.748+0000 7fef07fff700 1 -- 172.21.15.79:0/1289302742 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fef1808b350 con 0x7fef20156ae0 2023-06-23T18:36:33.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.748+0000 7fef1f7fe700 1 --2- 172.21.15.79:0/1289302742 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fef0c06ace0 0x7fef0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:33.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.748+0000 7fef1f7fe700 1 --2- 172.21.15.79:0/1289302742 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fef0c06ace0 0x7fef0c06d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fef14000c00 tx=0x7fef14008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:33.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.752+0000 7fef07fff700 1 -- 172.21.15.79:0/1289302742 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fef18056490 con 0x7fef20156ae0 2023-06-23T18:36:33.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.940+0000 7fef26017700 1 -- 172.21.15.79:0/1289302742 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fef20154fa0 con 0x7fef20156ae0 2023-06-23T18:36:33.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.940+0000 7fef07fff700 1 -- 172.21.15.79:0/1289302742 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fef18059ab0 con 0x7fef20156ae0 2023-06-23T18:36:33.945 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:33.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.942+0000 7fef26017700 1 -- 172.21.15.79:0/1289302742 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fef0c06ace0 msgr2=0x7fef0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.942+0000 7fef26017700 1 --2- 172.21.15.79:0/1289302742 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fef0c06ace0 0x7fef0c06d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fef14000c00 tx=0x7fef14008030 comp rx=0 tx=0).stop 2023-06-23T18:36:33.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.942+0000 7fef26017700 1 -- 172.21.15.79:0/1289302742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef20156ae0 msgr2=0x7fef200b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:33.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.942+0000 7fef26017700 1 --2- 172.21.15.79:0/1289302742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef20156ae0 0x7fef200b3cc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fef18007fc0 tx=0x7fef1800e100 comp rx=0 tx=0).stop 2023-06-23T18:36:33.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.942+0000 7fef26017700 1 -- 172.21.15.79:0/1289302742 shutdown_connections 2023-06-23T18:36:33.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.942+0000 7fef26017700 1 --2- 172.21.15.79:0/1289302742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef20156ae0 0x7fef200b3cc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.942+0000 7fef26017700 1 --2- 172.21.15.79:0/1289302742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef200cc060 0x7fef20154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.942+0000 7fef26017700 1 --2- 172.21.15.79:0/1289302742 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fef0c06ace0 0x7fef0c06d190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.942+0000 7fef26017700 1 --2- 172.21.15.79:0/1289302742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef200cc4b0 0x7fef20153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:33.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.942+0000 7fef26017700 1 -- 172.21.15.79:0/1289302742 >> 172.21.15.79:0/1289302742 conn(0x7fef2006afa0 msgr2=0x7fef2015acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:33.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.942+0000 7fef26017700 1 -- 172.21.15.79:0/1289302742 shutdown_connections 2023-06-23T18:36:33.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:33.942+0000 7fef26017700 1 -- 172.21.15.79:0/1289302742 wait complete. 2023-06-23T18:36:33.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:34.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.079+0000 7fd62664e700 1 Processor -- start 2023-06-23T18:36:34.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.079+0000 7fd62664e700 1 -- start start 2023-06-23T18:36:34.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.079+0000 7fd62664e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6200cd750 0x7fd620159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.079+0000 7fd62664e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd620158db0 0x7fd62015c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.079+0000 7fd62664e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62015d3c0 con 0x7fd62015be70 2023-06-23T18:36:34.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.079+0000 7fd62664e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62015c240 con 0x7fd620158db0 2023-06-23T18:36:34.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.079+0000 7fd62664e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd620159210 con 0x7fd6200cd750 2023-06-23T18:36:34.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.079+0000 7fd624beb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd620158db0 0x7fd62015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.080+0000 7fd61ffff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd62015be70 0x7fd6200cd040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43638/0 (socket says 172.21.15.79:43638) 2023-06-23T18:36:34.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.080+0000 7fd61ffff700 1 -- 172.21.15.79:0/104827497 learned_addr learned my addr 172.21.15.79:0/104827497 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:34.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.080+0000 7fd61f7fe700 1 --2- 172.21.15.79:0/104827497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6200cd750 0x7fd620159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.080+0000 7fd624beb700 1 -- 172.21.15.79:0/104827497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6200cd750 msgr2=0x7fd620159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:34.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.080+0000 7fd624beb700 1 --2- 172.21.15.79:0/104827497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6200cd750 0x7fd620159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.080+0000 7fd624beb700 1 -- 172.21.15.79:0/104827497 >> v1:172.21.15.79:6789/0 conn(0x7fd62015be70 legacy=0x7fd6200cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.080+0000 7fd624beb700 1 -- 172.21.15.79:0/104827497 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62006a0b0 con 0x7fd620158db0 2023-06-23T18:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.080+0000 7fd61effd700 1 -- 172.21.15.79:0/104827497 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36784808 0 0) 0x7fd62015d3c0 con 0x7fd62015be70 2023-06-23T18:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.080+0000 7fd624beb700 1 --2- 172.21.15.79:0/104827497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd620158db0 0x7fd62015c420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd618009d90 tx=0x7fd61800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81079bcdb0ef81cc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.080+0000 7fd61effd700 1 -- 172.21.15.79:0/104827497 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61800e070 con 0x7fd620158db0 2023-06-23T18:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.080+0000 7fd61effd700 1 -- 172.21.15.79:0/104827497 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61800bbd0 con 0x7fd620158db0 2023-06-23T18:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.080+0000 7fd61effd700 1 -- 172.21.15.79:0/104827497 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd618012430 con 0x7fd620158db0 2023-06-23T18:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.081+0000 7fd62664e700 1 -- 172.21.15.79:0/104827497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd620158db0 msgr2=0x7fd62015c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.081+0000 7fd62664e700 1 --2- 172.21.15.79:0/104827497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd620158db0 0x7fd62015c420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd618009d90 tx=0x7fd61800b330 comp rx=0 tx=0).stop 2023-06-23T18:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.081+0000 7fd62664e700 1 -- 172.21.15.79:0/104827497 shutdown_connections 2023-06-23T18:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.081+0000 7fd62664e700 1 --2- 172.21.15.79:0/104827497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd620158db0 0x7fd62015c420 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.081+0000 7fd62664e700 1 --2- 172.21.15.79:0/104827497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6200cd750 0x7fd620159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.081+0000 7fd62664e700 1 -- 172.21.15.79:0/104827497 >> 172.21.15.79:0/104827497 conn(0x7fd6200cfc60 msgr2=0x7fd6200b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.081+0000 7fd62664e700 1 -- 172.21.15.79:0/104827497 shutdown_connections 2023-06-23T18:36:34.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.081+0000 7fd62664e700 1 -- 172.21.15.79:0/104827497 wait complete. 2023-06-23T18:36:34.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.082+0000 7fd62664e700 1 Processor -- start 2023-06-23T18:36:34.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.083+0000 7fd62664e700 1 -- start start 2023-06-23T18:36:34.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.083+0000 7fd62664e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6200cd750 0x7fd6200c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.083+0000 7fd62664e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd620158db0 0x7fd6201e32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.083+0000 7fd62664e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd62015be70 0x7fd6201e3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.083+0000 7fd62664e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200c6e30 con 0x7fd620158db0 2023-06-23T18:36:34.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.083+0000 7fd62664e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200c6220 con 0x7fd6200cd750 2023-06-23T18:36:34.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.083+0000 7fd62664e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200c6500 con 0x7fd62015be70 2023-06-23T18:36:34.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.083+0000 7fd61f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd620158db0 0x7fd6201e32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd61f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd620158db0 0x7fd6201e32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40604/0 (socket says 172.21.15.79:40604) 2023-06-23T18:36:34.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd61f7fe700 1 -- 172.21.15.79:0/288564337 learned_addr learned my addr 172.21.15.79:0/288564337 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:34.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd61ffff700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6200cd750 0x7fd6200c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd624beb700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd62015be70 0x7fd6201e3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd61f7fe700 1 -- 172.21.15.79:0/288564337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd62015be70 msgr2=0x7fd6201e3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd61f7fe700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd62015be70 0x7fd6201e3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd61f7fe700 1 -- 172.21.15.79:0/288564337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6200cd750 msgr2=0x7fd6200c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd61f7fe700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6200cd750 0x7fd6200c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd61f7fe700 1 -- 172.21.15.79:0/288564337 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd618009a70 con 0x7fd620158db0 2023-06-23T18:36:34.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd624beb700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd62015be70 0x7fd6201e3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:34.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd61f7fe700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd620158db0 0x7fd6201e32d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd61400d800 tx=0x7fd61400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:34.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd61cff9700 1 -- 172.21.15.79:0/288564337 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140125f0 con 0x7fd620158db0 2023-06-23T18:36:34.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd61cff9700 1 -- 172.21.15.79:0/288564337 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd614012750 con 0x7fd620158db0 2023-06-23T18:36:34.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd61cff9700 1 -- 172.21.15.79:0/288564337 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614012ae0 con 0x7fd620158db0 2023-06-23T18:36:34.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd62664e700 1 -- 172.21.15.79:0/288564337 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6200c67e0 con 0x7fd620158db0 2023-06-23T18:36:34.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.084+0000 7fd62664e700 1 -- 172.21.15.79:0/288564337 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd620150dd0 con 0x7fd620158db0 2023-06-23T18:36:34.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.085+0000 7fd62664e700 1 -- 172.21.15.79:0/288564337 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6200672c0 con 0x7fd620158db0 2023-06-23T18:36:34.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.085+0000 7fd61cff9700 1 -- 172.21.15.79:0/288564337 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd614012c40 con 0x7fd620158db0 2023-06-23T18:36:34.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.086+0000 7fd61cff9700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd60c06ace0 0x7fd60c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.086+0000 7fd61cff9700 1 -- 172.21.15.79:0/288564337 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd6140895b0 con 0x7fd620158db0 2023-06-23T18:36:34.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.086+0000 7fd61ffff700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd60c06ace0 0x7fd60c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.086+0000 7fd61ffff700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd60c06ace0 0x7fd60c06d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd610007fd0 tx=0x7fd610007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:34.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.090+0000 7fd61cff9700 1 -- 172.21.15.79:0/288564337 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd614054660 con 0x7fd620158db0 2023-06-23T18:36:34.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.276+0000 7fd62664e700 1 -- 172.21.15.79:0/288564337 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd6200b1570 con 0x7fd620158db0 2023-06-23T18:36:34.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.276+0000 7fd61cff9700 1 -- 172.21.15.79:0/288564337 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd614057c80 con 0x7fd620158db0 2023-06-23T18:36:34.281 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:34.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.278+0000 7fd62664e700 1 -- 172.21.15.79:0/288564337 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd60c06ace0 msgr2=0x7fd60c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.278+0000 7fd62664e700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd60c06ace0 0x7fd60c06d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd610007fd0 tx=0x7fd610007de0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.278+0000 7fd62664e700 1 -- 172.21.15.79:0/288564337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd620158db0 msgr2=0x7fd6201e32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.278+0000 7fd62664e700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd620158db0 0x7fd6201e32d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd61400d800 tx=0x7fd61400db10 comp rx=0 tx=0).stop 2023-06-23T18:36:34.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.278+0000 7fd62664e700 1 -- 172.21.15.79:0/288564337 shutdown_connections 2023-06-23T18:36:34.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.278+0000 7fd62664e700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd62015be70 0x7fd6201e3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.278+0000 7fd62664e700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd60c06ace0 0x7fd60c06d190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.278+0000 7fd62664e700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6200cd750 0x7fd6200c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.278+0000 7fd62664e700 1 --2- 172.21.15.79:0/288564337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd620158db0 0x7fd6201e32d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.285 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.278+0000 7fd62664e700 1 -- 172.21.15.79:0/288564337 >> 172.21.15.79:0/288564337 conn(0x7fd6200cfc60 msgr2=0x7fd620159570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:34.285 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.279+0000 7fd62664e700 1 -- 172.21.15.79:0/288564337 shutdown_connections 2023-06-23T18:36:34.285 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.279+0000 7fd62664e700 1 -- 172.21.15.79:0/288564337 wait complete. 2023-06-23T18:36:34.295 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:34.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.413+0000 7ff19089d700 1 Processor -- start 2023-06-23T18:36:34.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.413+0000 7ff19089d700 1 -- start start 2023-06-23T18:36:34.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff19089d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff18c0cc060 0x7ff18c0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff19089d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff18c0cc9f0 con 0x7ff18c156ae0 2023-06-23T18:36:34.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff19089d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff18c15a560 con 0x7ff18c071710 2023-06-23T18:36:34.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff19089d700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff18c159950 con 0x7ff18c0cc060 2023-06-23T18:36:34.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff189d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff18c0cc060 0x7ff18c0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff18a59c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff18c156ae0 0x7ff18c15d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43652/0 (socket says 172.21.15.79:43652) 2023-06-23T18:36:34.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff18a59c700 1 -- 172.21.15.79:0/3851788269 learned_addr learned my addr 172.21.15.79:0/3851788269 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:34.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff189d9b700 1 -- 172.21.15.79:0/3851788269 >> v1:172.21.15.155:6789/0 conn(0x7ff18c071710 legacy=0x7ff18c071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff189d9b700 1 -- 172.21.15.79:0/3851788269 >> v1:172.21.15.79:6789/0 conn(0x7ff18c156ae0 legacy=0x7ff18c15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff189d9b700 1 -- 172.21.15.79:0/3851788269 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c159c30 con 0x7ff18c0cc060 2023-06-23T18:36:34.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff18959a700 1 -- 172.21.15.79:0/3851788269 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820722213 0 0) 0x7ff17c001110 con 0x7ff18c156ae0 2023-06-23T18:36:34.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff189d9b700 1 --2- 172.21.15.79:0/3851788269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff18c0cc060 0x7ff18c0cc4b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff180009840 tx=0x7ff180009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436e4f2270629197 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:34.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff18959a700 1 -- 172.21.15.79:0/3851788269 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff180011830 con 0x7ff18c0cc060 2023-06-23T18:36:34.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff18959a700 1 -- 172.21.15.79:0/3851788269 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff180011990 con 0x7ff18c0cc060 2023-06-23T18:36:34.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.414+0000 7ff18959a700 1 -- 172.21.15.79:0/3851788269 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff180011d20 con 0x7ff18c0cc060 2023-06-23T18:36:34.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.415+0000 7ff19089d700 1 -- 172.21.15.79:0/3851788269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff18c0cc060 msgr2=0x7ff18c0cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.415+0000 7ff19089d700 1 --2- 172.21.15.79:0/3851788269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff18c0cc060 0x7ff18c0cc4b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff180009840 tx=0x7ff180009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:34.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.415+0000 7ff19089d700 1 -- 172.21.15.79:0/3851788269 shutdown_connections 2023-06-23T18:36:34.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.415+0000 7ff19089d700 1 --2- 172.21.15.79:0/3851788269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff18c0cc060 0x7ff18c0cc4b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.415+0000 7ff19089d700 1 -- 172.21.15.79:0/3851788269 >> 172.21.15.79:0/3851788269 conn(0x7ff18c06afa0 msgr2=0x7ff18c06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:34.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.415+0000 7ff19089d700 1 -- 172.21.15.79:0/3851788269 shutdown_connections 2023-06-23T18:36:34.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.415+0000 7ff19089d700 1 -- 172.21.15.79:0/3851788269 wait complete. 2023-06-23T18:36:34.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.416+0000 7ff19089d700 1 Processor -- start 2023-06-23T18:36:34.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.416+0000 7ff19089d700 1 -- start start 2023-06-23T18:36:34.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff19089d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff18c071710 0x7ff18c1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff19089d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0cc060 0x7ff18c1531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff19089d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff18c156ae0 0x7ff18c0b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff19089d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0b0610 con 0x7ff18c156ae0 2023-06-23T18:36:34.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff19089d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0afa20 con 0x7ff18c071710 2023-06-23T18:36:34.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff19089d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0b0f70 con 0x7ff18c0cc060 2023-06-23T18:36:34.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff18ad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff18c156ae0 0x7ff18c0b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff18ad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff18c156ae0 0x7ff18c0b0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40620/0 (socket says 172.21.15.79:40620) 2023-06-23T18:36:34.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff18ad9d700 1 -- 172.21.15.79:0/1483298061 learned_addr learned my addr 172.21.15.79:0/1483298061 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:34.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff18a59c700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff18c071710 0x7ff18c1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff189d9b700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0cc060 0x7ff18c1531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff18ad9d700 1 -- 172.21.15.79:0/1483298061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0cc060 msgr2=0x7ff18c1531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff18ad9d700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0cc060 0x7ff18c1531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff18ad9d700 1 -- 172.21.15.79:0/1483298061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff18c071710 msgr2=0x7ff18c1537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.417+0000 7ff18ad9d700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff18c071710 0x7ff18c1537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.418+0000 7ff18ad9d700 1 -- 172.21.15.79:0/1483298061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1800094a0 con 0x7ff18c156ae0 2023-06-23T18:36:34.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.418+0000 7ff189d9b700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0cc060 0x7ff18c1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:34.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.418+0000 7ff18ad9d700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff18c156ae0 0x7ff18c0b0a30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff18400cc50 tx=0x7ff18400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:34.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.418+0000 7ff17affd700 1 -- 172.21.15.79:0/1483298061 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff184003c00 con 0x7ff18c156ae0 2023-06-23T18:36:34.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.418+0000 7ff17affd700 1 -- 172.21.15.79:0/1483298061 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff184003d60 con 0x7ff18c156ae0 2023-06-23T18:36:34.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.418+0000 7ff19089d700 1 -- 172.21.15.79:0/1483298061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18c0afc70 con 0x7ff18c156ae0 2023-06-23T18:36:34.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.418+0000 7ff17affd700 1 -- 172.21.15.79:0/1483298061 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18400ab60 con 0x7ff18c156ae0 2023-06-23T18:36:34.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.418+0000 7ff19089d700 1 -- 172.21.15.79:0/1483298061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18c0b0230 con 0x7ff18c156ae0 2023-06-23T18:36:34.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.419+0000 7ff19089d700 1 -- 172.21.15.79:0/1483298061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18c0b0750 con 0x7ff18c156ae0 2023-06-23T18:36:34.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.419+0000 7ff17affd700 1 -- 172.21.15.79:0/1483298061 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff18400acc0 con 0x7ff18c156ae0 2023-06-23T18:36:34.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.420+0000 7ff17affd700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff16c06ace0 0x7ff16c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.420+0000 7ff18a59c700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff16c06ace0 0x7ff16c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.420+0000 7ff17affd700 1 -- 172.21.15.79:0/1483298061 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff1840894b0 con 0x7ff18c156ae0 2023-06-23T18:36:34.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.420+0000 7ff18a59c700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff16c06ace0 0x7ff16c06d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff17c007c40 tx=0x7ff17c007b90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:34.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.424+0000 7ff17affd700 1 -- 172.21.15.79:0/1483298061 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff18c0b0750 con 0x7ff18c156ae0 2023-06-23T18:36:34.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.610+0000 7ff19089d700 1 -- 172.21.15.79:0/1483298061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff18c0b0750 con 0x7ff18c156ae0 2023-06-23T18:36:34.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.611+0000 7ff17affd700 1 -- 172.21.15.79:0/1483298061 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff18c0b0750 con 0x7ff18c156ae0 2023-06-23T18:36:34.616 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:34.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.612+0000 7ff19089d700 1 -- 172.21.15.79:0/1483298061 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff16c06ace0 msgr2=0x7ff16c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.612+0000 7ff19089d700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff16c06ace0 0x7ff16c06d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff17c007c40 tx=0x7ff17c007b90 comp rx=0 tx=0).stop 2023-06-23T18:36:34.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.612+0000 7ff19089d700 1 -- 172.21.15.79:0/1483298061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff18c156ae0 msgr2=0x7ff18c0b0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.612+0000 7ff19089d700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff18c156ae0 0x7ff18c0b0a30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff18400cc50 tx=0x7ff18400cf60 comp rx=0 tx=0).stop 2023-06-23T18:36:34.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.612+0000 7ff19089d700 1 -- 172.21.15.79:0/1483298061 shutdown_connections 2023-06-23T18:36:34.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.612+0000 7ff19089d700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff18c156ae0 0x7ff18c0b0a30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.612+0000 7ff19089d700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff16c06ace0 0x7ff16c06d190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.612+0000 7ff19089d700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff18c071710 0x7ff18c1537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.612+0000 7ff19089d700 1 --2- 172.21.15.79:0/1483298061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0cc060 0x7ff18c1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.612+0000 7ff19089d700 1 -- 172.21.15.79:0/1483298061 >> 172.21.15.79:0/1483298061 conn(0x7ff18c06afa0 msgr2=0x7ff18c0cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:34.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.613+0000 7ff19089d700 1 -- 172.21.15.79:0/1483298061 shutdown_connections 2023-06-23T18:36:34.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.613+0000 7ff19089d700 1 -- 172.21.15.79:0/1483298061 wait complete. 2023-06-23T18:36:34.629 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:36:34.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.745+0000 7f78c5bf6700 1 Processor -- start 2023-06-23T18:36:34.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.745+0000 7f78c5bf6700 1 -- start start 2023-06-23T18:36:34.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.746+0000 7f78c5bf6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c00b1270 0x7f78c015c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.746+0000 7f78c5bf6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f78c00aee30 0x7f78c015cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.746+0000 7f78c5bf6700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c01608f0 con 0x7f78c0071490 2023-06-23T18:36:34.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.746+0000 7f78c5bf6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c0160e20 con 0x7f78c00aee30 2023-06-23T18:36:34.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.746+0000 7f78c5bf6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c00cd4f0 con 0x7f78c00b1270 2023-06-23T18:36:34.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.746+0000 7f78bffff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f78c00aee30 0x7f78c015cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.746+0000 7f78bf7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f78c0071490 0x7f78c0071860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43666/0 (socket says 172.21.15.79:43666) 2023-06-23T18:36:34.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.746+0000 7f78bf7fe700 1 -- 172.21.15.79:0/4112563944 learned_addr learned my addr 172.21.15.79:0/4112563944 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:34.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.746+0000 7f78bffff700 1 -- 172.21.15.79:0/4112563944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c00b1270 msgr2=0x7f78c015c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:34.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.746+0000 7f78bffff700 1 --2- 172.21.15.79:0/4112563944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c00b1270 0x7f78c015c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.746+0000 7f78bffff700 1 -- 172.21.15.79:0/4112563944 >> v1:172.21.15.79:6789/0 conn(0x7f78c0071490 legacy=0x7f78c0071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.746+0000 7f78be7fc700 1 -- 172.21.15.79:0/4112563944 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278965543 0 0) 0x7f78c01608f0 con 0x7f78c0071490 2023-06-23T18:36:34.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.746+0000 7f78bffff700 1 -- 172.21.15.79:0/4112563944 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c0160b40 con 0x7f78c00aee30 2023-06-23T18:36:34.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.746+0000 7f78bffff700 1 --2- 172.21.15.79:0/4112563944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78c00aee30 0x7f78c015cf30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009d90 tx=0x7f78b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e5c52a52533b1e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:34.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.747+0000 7f78be7fc700 1 -- 172.21.15.79:0/4112563944 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b800e070 con 0x7f78c00aee30 2023-06-23T18:36:34.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.747+0000 7f78be7fc700 1 -- 172.21.15.79:0/4112563944 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b800bbd0 con 0x7f78c00aee30 2023-06-23T18:36:34.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.747+0000 7f78be7fc700 1 -- 172.21.15.79:0/4112563944 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8012430 con 0x7f78c00aee30 2023-06-23T18:36:34.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.747+0000 7f78c5bf6700 1 -- 172.21.15.79:0/4112563944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78c00aee30 msgr2=0x7f78c015cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.747+0000 7f78c5bf6700 1 --2- 172.21.15.79:0/4112563944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78c00aee30 0x7f78c015cf30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009d90 tx=0x7f78b800b330 comp rx=0 tx=0).stop 2023-06-23T18:36:34.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.747+0000 7f78c5bf6700 1 -- 172.21.15.79:0/4112563944 shutdown_connections 2023-06-23T18:36:34.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.747+0000 7f78c5bf6700 1 --2- 172.21.15.79:0/4112563944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78c00aee30 0x7f78c015cf30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.747+0000 7f78c5bf6700 1 --2- 172.21.15.79:0/4112563944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c00b1270 0x7f78c015c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.747+0000 7f78c5bf6700 1 -- 172.21.15.79:0/4112563944 >> 172.21.15.79:0/4112563944 conn(0x7f78c006d660 msgr2=0x7f78c006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:34.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.747+0000 7f78c5bf6700 1 -- 172.21.15.79:0/4112563944 shutdown_connections 2023-06-23T18:36:34.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.747+0000 7f78c5bf6700 1 -- 172.21.15.79:0/4112563944 wait complete. 2023-06-23T18:36:34.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.748+0000 7f78c5bf6700 1 Processor -- start 2023-06-23T18:36:34.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.749+0000 7f78c5bf6700 1 -- start start 2023-06-23T18:36:34.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.749+0000 7f78c5bf6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c0071490 0x7f78c00c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.749+0000 7f78c5bf6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c00aee30 0x7f78c00ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.749+0000 7f78c5bf6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78c00b1270 0x7f78c00cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.749+0000 7f78c5bf6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c00c4820 con 0x7f78c00b1270 2023-06-23T18:36:34.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.749+0000 7f78c5bf6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c00c3ba0 con 0x7f78c00aee30 2023-06-23T18:36:34.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.749+0000 7f78c5bf6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c00c3e80 con 0x7f78c0071490 2023-06-23T18:36:34.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.749+0000 7f78bffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78c00b1270 0x7f78c00cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.749+0000 7f78bffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78c00b1270 0x7f78c00cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40634/0 (socket says 172.21.15.79:40634) 2023-06-23T18:36:34.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.749+0000 7f78bffff700 1 -- 172.21.15.79:0/1427130296 learned_addr learned my addr 172.21.15.79:0/1427130296 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:34.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.749+0000 7f78bf7fe700 1 --2- 172.21.15.79:0/1427130296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c0071490 0x7f78c00c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.749+0000 7f78beffd700 1 --2- 172.21.15.79:0/1427130296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c00aee30 0x7f78c00ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.750+0000 7f78bffff700 1 -- 172.21.15.79:0/1427130296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c0071490 msgr2=0x7f78c00c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.750+0000 7f78bffff700 1 --2- 172.21.15.79:0/1427130296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c0071490 0x7f78c00c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.750+0000 7f78bffff700 1 -- 172.21.15.79:0/1427130296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c00aee30 msgr2=0x7f78c00ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.750+0000 7f78bffff700 1 --2- 172.21.15.79:0/1427130296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c00aee30 0x7f78c00ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.750+0000 7f78bffff700 1 -- 172.21.15.79:0/1427130296 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b8009a70 con 0x7f78c00b1270 2023-06-23T18:36:34.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.750+0000 7f78bffff700 1 --2- 172.21.15.79:0/1427130296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78c00b1270 0x7f78c00cbd30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f78b800b370 tx=0x7f78b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:34.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.750+0000 7f78a7fff700 1 -- 172.21.15.79:0/1427130296 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8003ea0 con 0x7f78c00b1270 2023-06-23T18:36:34.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.750+0000 7f78a7fff700 1 -- 172.21.15.79:0/1427130296 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b8020070 con 0x7f78c00b1270 2023-06-23T18:36:34.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.750+0000 7f78c5bf6700 1 -- 172.21.15.79:0/1427130296 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c00c4160 con 0x7f78c00b1270 2023-06-23T18:36:34.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.750+0000 7f78a7fff700 1 -- 172.21.15.79:0/1427130296 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8004280 con 0x7f78c00b1270 2023-06-23T18:36:34.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.750+0000 7f78c5bf6700 1 -- 172.21.15.79:0/1427130296 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c00c3830 con 0x7f78c00b1270 2023-06-23T18:36:34.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.751+0000 7f78c5bf6700 1 -- 172.21.15.79:0/1427130296 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c00724a0 con 0x7f78c00b1270 2023-06-23T18:36:34.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.751+0000 7f78a7fff700 1 -- 172.21.15.79:0/1427130296 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f78b80043e0 con 0x7f78c00b1270 2023-06-23T18:36:34.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.752+0000 7f78a7fff700 1 --2- 172.21.15.79:0/1427130296 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f78ac06ace0 0x7f78ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:34.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.752+0000 7f78a7fff700 1 -- 172.21.15.79:0/1427130296 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f78b8022070 con 0x7f78c00b1270 2023-06-23T18:36:34.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.752+0000 7f78bf7fe700 1 --2- 172.21.15.79:0/1427130296 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f78ac06ace0 0x7f78ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:34.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.752+0000 7f78bf7fe700 1 --2- 172.21.15.79:0/1427130296 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f78ac06ace0 0x7f78ac06d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f78b4000f60 tx=0x7f78b4007e70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:34.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.756+0000 7f78a7fff700 1 -- 172.21.15.79:0/1427130296 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f78b8056360 con 0x7f78c00b1270 2023-06-23T18:36:34.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.955+0000 7f78c5bf6700 1 -- 172.21.15.79:0/1427130296 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f78c00b5770 con 0x7f78c00b1270 2023-06-23T18:36:34.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.956+0000 7f78a7fff700 1 -- 172.21.15.79:0/1427130296 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f78b8012e60 con 0x7f78c00b1270 2023-06-23T18:36:34.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.958+0000 7f78c5bf6700 1 -- 172.21.15.79:0/1427130296 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f78ac06ace0 msgr2=0x7f78ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.958+0000 7f78c5bf6700 1 --2- 172.21.15.79:0/1427130296 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f78ac06ace0 0x7f78ac06d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f78b4000f60 tx=0x7f78b4007e70 comp rx=0 tx=0).stop 2023-06-23T18:36:34.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.958+0000 7f78c5bf6700 1 -- 172.21.15.79:0/1427130296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78c00b1270 msgr2=0x7f78c00cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:34.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.958+0000 7f78c5bf6700 1 --2- 172.21.15.79:0/1427130296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78c00b1270 0x7f78c00cbd30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f78b800b370 tx=0x7f78b8004ad0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.958+0000 7f78c5bf6700 1 -- 172.21.15.79:0/1427130296 shutdown_connections 2023-06-23T18:36:34.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.958+0000 7f78c5bf6700 1 --2- 172.21.15.79:0/1427130296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78c00b1270 0x7f78c00cbd30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.958+0000 7f78c5bf6700 1 --2- 172.21.15.79:0/1427130296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c0071490 0x7f78c00c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.958+0000 7f78c5bf6700 1 --2- 172.21.15.79:0/1427130296 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f78ac06ace0 0x7f78ac06d190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.958+0000 7f78c5bf6700 1 --2- 172.21.15.79:0/1427130296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c00aee30 0x7f78c00ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:34.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.958+0000 7f78c5bf6700 1 -- 172.21.15.79:0/1427130296 >> 172.21.15.79:0/1427130296 conn(0x7f78c006d660 msgr2=0x7f78c0155ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:34.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.958+0000 7f78c5bf6700 1 -- 172.21.15.79:0/1427130296 shutdown_connections 2023-06-23T18:36:34.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:34.958+0000 7f78c5bf6700 1 -- 172.21.15.79:0/1427130296 wait complete. 2023-06-23T18:36:34.965 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-23T18:36:34.975 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-06-23T18:36:34.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:34.976 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-06-23T18:36:35.019 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:36:35.019 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:36:35.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:35.020 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-06-23T18:36:35.086 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:36:35.086 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:36:35.103 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:36:35.104 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:36:35.160 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:35.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.315+0000 7f7b7c56a700 1 Processor -- start 2023-06-23T18:36:35.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.315+0000 7f7b7c56a700 1 -- start start 2023-06-23T18:36:35.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.316+0000 7f7b7c56a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b740b1f40 0x7f7b74069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.316+0000 7f7b7c56a700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7b7406a3d0 0x7f7b740cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.316+0000 7f7b7c56a700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b740ca550 con 0x7f7b740b4f20 2023-06-23T18:36:35.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.316+0000 7f7b7c56a700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b7406a850 con 0x7f7b7406a3d0 2023-06-23T18:36:35.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.316+0000 7f7b7c56a700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7415a810 con 0x7f7b740b1f40 2023-06-23T18:36:35.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.317+0000 7f7b7ab07700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7b7406a3d0 0x7f7b740cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:35.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.317+0000 7f7b79b05700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b740b1f40 0x7f7b74069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:35.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.317+0000 7f7b7ab07700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7b7406a3d0 0x7f7b740cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:46868/0 (socket says 172.21.15.79:46868) 2023-06-23T18:36:35.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.317+0000 7f7b7ab07700 1 -- 172.21.15.79:0/1784612685 learned_addr learned my addr 172.21.15.79:0/1784612685 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:35.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.317+0000 7f7b7a306700 1 --1- 172.21.15.79:0/1784612685 >> v1:172.21.15.155:6789/0 conn(0x7f7b740b4f20 0x7f7b740c9e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:51948/0 (socket says 172.21.15.79:51948) 2023-06-23T18:36:35.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.317+0000 7f7b79b05700 1 -- 172.21.15.79:0/1784612685 >> v2:172.21.15.155:3300/0 conn(0x7f7b7406a3d0 msgr2=0x7f7b740cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:35.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.317+0000 7f7b79b05700 1 --2- 172.21.15.79:0/1784612685 >> v2:172.21.15.155:3300/0 conn(0x7f7b7406a3d0 0x7f7b740cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.317+0000 7f7b79b05700 1 -- 172.21.15.79:0/1784612685 >> v1:172.21.15.155:6789/0 conn(0x7f7b740b4f20 legacy=0x7f7b740c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:35.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.317+0000 7f7b79b05700 1 -- 172.21.15.79:0/1784612685 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b740b52f0 con 0x7f7b740b1f40 2023-06-23T18:36:35.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.317+0000 7f7b7ab07700 1 --2- 172.21.15.79:0/1784612685 >> v2:172.21.15.155:3300/0 conn(0x7f7b7406a3d0 0x7f7b740cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:35.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.317+0000 7f7b79b05700 1 --2- 172.21.15.79:0/1784612685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b740b1f40 0x7f7b74069e90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009840 tx=0x7f7b70009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ae29f293cf49bba server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:35.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.318+0000 7f7b79304700 1 -- 172.21.15.79:0/1784612685 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b70011830 con 0x7f7b740b1f40 2023-06-23T18:36:35.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.318+0000 7f7b79304700 1 -- 172.21.15.79:0/1784612685 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b70011990 con 0x7f7b740b1f40 2023-06-23T18:36:35.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.318+0000 7f7b79304700 1 -- 172.21.15.79:0/1784612685 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b70011d20 con 0x7f7b740b1f40 2023-06-23T18:36:35.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.318+0000 7f7b7c56a700 1 -- 172.21.15.79:0/1784612685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b740b1f40 msgr2=0x7f7b74069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:35.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.318+0000 7f7b7c56a700 1 --2- 172.21.15.79:0/1784612685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b740b1f40 0x7f7b74069e90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009840 tx=0x7f7b70009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:35.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.318+0000 7f7b7c56a700 1 -- 172.21.15.79:0/1784612685 shutdown_connections 2023-06-23T18:36:35.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.318+0000 7f7b7c56a700 1 --2- 172.21.15.79:0/1784612685 >> v2:172.21.15.155:3300/0 conn(0x7f7b7406a3d0 0x7f7b740cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.318+0000 7f7b7c56a700 1 --2- 172.21.15.79:0/1784612685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b740b1f40 0x7f7b74069e90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.318+0000 7f7b7c56a700 1 -- 172.21.15.79:0/1784612685 >> 172.21.15.79:0/1784612685 conn(0x7f7b7406d660 msgr2=0x7f7b7406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:35.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.318+0000 7f7b7c56a700 1 -- 172.21.15.79:0/1784612685 shutdown_connections 2023-06-23T18:36:35.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.318+0000 7f7b7c56a700 1 -- 172.21.15.79:0/1784612685 wait complete. 2023-06-23T18:36:35.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.319+0000 7f7b7c56a700 1 Processor -- start 2023-06-23T18:36:35.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.319+0000 7f7b7c56a700 1 -- start start 2023-06-23T18:36:35.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.320+0000 7f7b7c56a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b7406a3d0 0x7f7b740cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.320+0000 7f7b7c56a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b740b1f40 0x7f7b74156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.320+0000 7f7b7c56a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b740b4f20 0x7f7b740ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.320+0000 7f7b7c56a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740c7b70 con 0x7f7b740b1f40 2023-06-23T18:36:35.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.320+0000 7f7b7c56a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741f4090 con 0x7f7b740b4f20 2023-06-23T18:36:35.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.320+0000 7f7b7c56a700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741f4370 con 0x7f7b7406a3d0 2023-06-23T18:36:35.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.320+0000 7f7b79b05700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b740b1f40 0x7f7b74156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:35.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.320+0000 7f7b79b05700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b740b1f40 0x7f7b74156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40646/0 (socket says 172.21.15.79:40646) 2023-06-23T18:36:35.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.320+0000 7f7b79b05700 1 -- 172.21.15.79:0/2715050341 learned_addr learned my addr 172.21.15.79:0/2715050341 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:35.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.320+0000 7f7b7a306700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b7406a3d0 0x7f7b740cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:35.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.320+0000 7f7b7ab07700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b740b4f20 0x7f7b740ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:35.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.321+0000 7f7b79b05700 1 -- 172.21.15.79:0/2715050341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b7406a3d0 msgr2=0x7f7b740cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:35.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.321+0000 7f7b79b05700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b7406a3d0 0x7f7b740cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.321+0000 7f7b79b05700 1 -- 172.21.15.79:0/2715050341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b740b4f20 msgr2=0x7f7b740ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:35.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.321+0000 7f7b79b05700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b740b4f20 0x7f7b740ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.321+0000 7f7b79b05700 1 -- 172.21.15.79:0/2715050341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c009b30 con 0x7f7b740b1f40 2023-06-23T18:36:35.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.321+0000 7f7b7ab07700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b740b4f20 0x7f7b740ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:35.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.321+0000 7f7b79b05700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b740b1f40 0x7f7b74156950 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009fd0 tx=0x7f7b70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:35.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.321+0000 7f7b6affd700 1 -- 172.21.15.79:0/2715050341 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b700045d0 con 0x7f7b740b1f40 2023-06-23T18:36:35.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.321+0000 7f7b6affd700 1 -- 172.21.15.79:0/2715050341 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7001f070 con 0x7f7b740b1f40 2023-06-23T18:36:35.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.321+0000 7f7b7c56a700 1 -- 172.21.15.79:0/2715050341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b700094a0 con 0x7f7b740b1f40 2023-06-23T18:36:35.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.321+0000 7f7b6affd700 1 -- 172.21.15.79:0/2715050341 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b700038e0 con 0x7f7b740b1f40 2023-06-23T18:36:35.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.321+0000 7f7b7c56a700 1 -- 172.21.15.79:0/2715050341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b741f4930 con 0x7f7b740b1f40 2023-06-23T18:36:35.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.322+0000 7f7b7c56a700 1 -- 172.21.15.79:0/2715050341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b740c7cb0 con 0x7f7b740b1f40 2023-06-23T18:36:35.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.322+0000 7f7b6affd700 1 -- 172.21.15.79:0/2715050341 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7b70003a40 con 0x7f7b740b1f40 2023-06-23T18:36:35.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.323+0000 7f7b6affd700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7b5806ace0 0x7f7b5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.323+0000 7f7b6affd700 1 -- 172.21.15.79:0/2715050341 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7b70021070 con 0x7f7b740b1f40 2023-06-23T18:36:35.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.323+0000 7f7b7a306700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7b5806ace0 0x7f7b5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:35.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.323+0000 7f7b7a306700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7b5806ace0 0x7f7b5806d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7b64000f60 tx=0x7f7b64007f10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:35.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.327+0000 7f7b6affd700 1 -- 172.21.15.79:0/2715050341 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7b70055dd0 con 0x7f7b740b1f40 2023-06-23T18:36:35.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.513+0000 7f7b7c56a700 1 -- 172.21.15.79:0/2715050341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b7404e240 con 0x7f7b740b1f40 2023-06-23T18:36:35.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.514+0000 7f7b6affd700 1 -- 172.21.15.79:0/2715050341 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7f7b70015090 con 0x7f7b740b1f40 2023-06-23T18:36:35.519 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:35.519 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:35.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.516+0000 7f7b7c56a700 1 -- 172.21.15.79:0/2715050341 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7b5806ace0 msgr2=0x7f7b5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:35.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.516+0000 7f7b7c56a700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7b5806ace0 0x7f7b5806d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7b64000f60 tx=0x7f7b64007f10 comp rx=0 tx=0).stop 2023-06-23T18:36:35.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.516+0000 7f7b7c56a700 1 -- 172.21.15.79:0/2715050341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b740b1f40 msgr2=0x7f7b74156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:35.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.516+0000 7f7b7c56a700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b740b1f40 0x7f7b74156950 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009fd0 tx=0x7f7b70004750 comp rx=0 tx=0).stop 2023-06-23T18:36:35.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.516+0000 7f7b7c56a700 1 -- 172.21.15.79:0/2715050341 shutdown_connections 2023-06-23T18:36:35.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.516+0000 7f7b7c56a700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7b5806ace0 0x7f7b5806d190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.516+0000 7f7b7c56a700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b740b4f20 0x7f7b740ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.516+0000 7f7b7c56a700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b7406a3d0 0x7f7b740cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.516+0000 7f7b7c56a700 1 --2- 172.21.15.79:0/2715050341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b740b1f40 0x7f7b74156950 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.516+0000 7f7b7c56a700 1 -- 172.21.15.79:0/2715050341 >> 172.21.15.79:0/2715050341 conn(0x7f7b7406d660 msgr2=0x7f7b740afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:35.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.516+0000 7f7b7c56a700 1 -- 172.21.15.79:0/2715050341 shutdown_connections 2023-06-23T18:36:35.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.516+0000 7f7b7c56a700 1 -- 172.21.15.79:0/2715050341 wait complete. 2023-06-23T18:36:35.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:35.534 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:35.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.651+0000 7f9e5a6dd700 1 Processor -- start 2023-06-23T18:36:35.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.651+0000 7f9e5a6dd700 1 -- start start 2023-06-23T18:36:35.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.652+0000 7f9e5a6dd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e5415d750 0x7f9e5415e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.652+0000 7f9e5a6dd700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e54071cf0 con 0x7f9e54072180 2023-06-23T18:36:35.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.652+0000 7f9e5a6dd700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5415ecf0 con 0x7f9e5406a5a0 2023-06-23T18:36:35.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.652+0000 7f9e5a6dd700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5415db20 con 0x7f9e5415d750 2023-06-23T18:36:35.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.652+0000 7f9e537fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9e54072180 0x7f9e54069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43696/0 (socket says 172.21.15.79:43696) 2023-06-23T18:36:35.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.652+0000 7f9e537fe700 1 -- 172.21.15.79:0/2559826365 learned_addr learned my addr 172.21.15.79:0/2559826365 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:35.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e53fff700 1 --2- 172.21.15.79:0/2559826365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e5415d750 0x7f9e5415e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:35.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2765697042 0 0) 0x7f9e54071cf0 con 0x7f9e54072180 2023-06-23T18:36:35.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e3c0035b0 con 0x7f9e54072180 2023-06-23T18:36:35.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884950384 0 0) 0x7f9e5415ecf0 con 0x7f9e5406a5a0 2023-06-23T18:36:35.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e54071cf0 con 0x7f9e5406a5a0 2023-06-23T18:36:35.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2632405454 0 0) 0x7f9e3c0035b0 con 0x7f9e54072180 2023-06-23T18:36:35.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e5415d750 msgr2=0x7f9e5415e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:35.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 --2- 172.21.15.79:0/2559826365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e5415d750 0x7f9e5415e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 >> v1:172.21.15.155:6789/0 conn(0x7f9e5406a5a0 legacy=0x7f9e540b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:35.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e540ca310 con 0x7f9e54072180 2023-06-23T18:36:35.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f9e44001670 con 0x7f9e54072180 2023-06-23T18:36:35.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 >> v1:172.21.15.79:6789/0 conn(0x7f9e54072180 legacy=0x7f9e54069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:35.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 --2- 172.21.15.79:0/2559826365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e3c003bc0 0x7f9e3c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 --2- 172.21.15.79:0/2559826365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e3c004850 0x7f9e3c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 --2- 172.21.15.79:0/2559826365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e3c005250 0x7f9e3c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5415db20 con 0x7f9e3c004850 2023-06-23T18:36:35.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c00c160 con 0x7f9e3c005250 2023-06-23T18:36:35.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c00c320 con 0x7f9e3c003bc0 2023-06-23T18:36:35.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e53fff700 1 --2- 172.21.15.79:0/2559826365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e5415d750 0x7f9e5415e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:35.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e537fe700 1 --2- 172.21.15.79:0/2559826365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e3c004850 0x7f9e3c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:35.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e53fff700 1 --2- 172.21.15.79:0/2559826365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e3c003bc0 0x7f9e3c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:35.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e58c7a700 1 --2- 172.21.15.79:0/2559826365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e3c005250 0x7f9e3c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:35.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e537fe700 1 -- 172.21.15.79:0/2559826365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e3c003bc0 msgr2=0x7f9e3c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:35.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e537fe700 1 --2- 172.21.15.79:0/2559826365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e3c003bc0 0x7f9e3c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e537fe700 1 -- 172.21.15.79:0/2559826365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e3c005250 msgr2=0x7f9e3c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:35.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e537fe700 1 --2- 172.21.15.79:0/2559826365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e3c005250 0x7f9e3c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.653+0000 7f9e537fe700 1 -- 172.21.15.79:0/2559826365 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0035b0 con 0x7f9e3c004850 2023-06-23T18:36:35.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.654+0000 7f9e537fe700 1 --2- 172.21.15.79:0/2559826365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e3c004850 0x7f9e3c004cc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9e44001eb0 tx=0x7f9e3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d1dd2760d5873c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:35.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.654+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4400cc10 con 0x7f9e3c004850 2023-06-23T18:36:35.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.654+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4400cd70 con 0x7f9e3c004850 2023-06-23T18:36:35.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.654+0000 7f9e52ffd700 1 -- 172.21.15.79:0/2559826365 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e44005ba0 con 0x7f9e3c004850 2023-06-23T18:36:35.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.654+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2559826365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e3c004850 msgr2=0x7f9e3c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:35.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.654+0000 7f9e5a6dd700 1 --2- 172.21.15.79:0/2559826365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e3c004850 0x7f9e3c004cc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9e44001eb0 tx=0x7f9e3c005880 comp rx=0 tx=0).stop 2023-06-23T18:36:35.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.654+0000 7f9e58c7a700 1 -- 172.21.15.79:0/2559826365 reap_dead start 2023-06-23T18:36:35.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.654+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2559826365 shutdown_connections 2023-06-23T18:36:35.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.654+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2559826365 >> 172.21.15.79:0/2559826365 conn(0x7f9e5406afa0 msgr2=0x7f9e5406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:35.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.654+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2559826365 shutdown_connections 2023-06-23T18:36:35.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.654+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2559826365 wait complete. 2023-06-23T18:36:35.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.655+0000 7f9e5a6dd700 1 Processor -- start 2023-06-23T18:36:35.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.656+0000 7f9e5a6dd700 1 -- start start 2023-06-23T18:36:35.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.656+0000 7f9e5a6dd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e3c004850 0x7f9e540b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.656+0000 7f9e5a6dd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e540c6e00 0x7f9e540c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.656+0000 7f9e5a6dd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e540b1910 0x7f9e540c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.656+0000 7f9e5a6dd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e540c7290 con 0x7f9e540b1910 2023-06-23T18:36:35.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.656+0000 7f9e5a6dd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e540cd5c0 con 0x7f9e3c004850 2023-06-23T18:36:35.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.656+0000 7f9e5a6dd700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e540cd8a0 con 0x7f9e540c6e00 2023-06-23T18:36:35.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e58c7a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e540b1910 0x7f9e540c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:35.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e58c7a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e540b1910 0x7f9e540c88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40668/0 (socket says 172.21.15.79:40668) 2023-06-23T18:36:35.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e58c7a700 1 -- 172.21.15.79:0/2051031354 learned_addr learned my addr 172.21.15.79:0/2051031354 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:35.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e53fff700 1 --2- 172.21.15.79:0/2051031354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e3c004850 0x7f9e540b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:35.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e537fe700 1 --2- 172.21.15.79:0/2051031354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e540c6e00 0x7f9e540c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:35.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e58c7a700 1 -- 172.21.15.79:0/2051031354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e540c6e00 msgr2=0x7f9e540c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:35.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e58c7a700 1 --2- 172.21.15.79:0/2051031354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e540c6e00 0x7f9e540c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e58c7a700 1 -- 172.21.15.79:0/2051031354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e3c004850 msgr2=0x7f9e540b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:35.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e58c7a700 1 --2- 172.21.15.79:0/2051031354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e3c004850 0x7f9e540b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e58c7a700 1 -- 172.21.15.79:0/2051031354 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e540cdb80 con 0x7f9e540b1910 2023-06-23T18:36:35.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e58c7a700 1 --2- 172.21.15.79:0/2051031354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e540b1910 0x7f9e540c88a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9e54071270 tx=0x7f9e4c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:35.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e50ff9700 1 -- 172.21.15.79:0/2051031354 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c003c10 con 0x7f9e540b1910 2023-06-23T18:36:35.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e50ff9700 1 -- 172.21.15.79:0/2051031354 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c003d70 con 0x7f9e540b1910 2023-06-23T18:36:35.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2051031354 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e54155110 con 0x7f9e540b1910 2023-06-23T18:36:35.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e50ff9700 1 -- 172.21.15.79:0/2051031354 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c005310 con 0x7f9e540b1910 2023-06-23T18:36:35.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.657+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2051031354 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e541f5800 con 0x7f9e540b1910 2023-06-23T18:36:35.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.658+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2051031354 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e540672c0 con 0x7f9e540b1910 2023-06-23T18:36:35.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.659+0000 7f9e50ff9700 1 -- 172.21.15.79:0/2051031354 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9e4c005470 con 0x7f9e540b1910 2023-06-23T18:36:35.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.659+0000 7f9e50ff9700 1 --2- 172.21.15.79:0/2051031354 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9e4006ace0 0x7f9e4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.659+0000 7f9e50ff9700 1 -- 172.21.15.79:0/2051031354 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e4c089380 con 0x7f9e540b1910 2023-06-23T18:36:35.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.659+0000 7f9e53fff700 1 --2- 172.21.15.79:0/2051031354 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9e4006ace0 0x7f9e4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:35.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.659+0000 7f9e53fff700 1 --2- 172.21.15.79:0/2051031354 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9e4006ace0 0x7f9e4006d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9e5415d120 tx=0x7f9e48008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:35.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.663+0000 7f9e50ff9700 1 -- 172.21.15.79:0/2051031354 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9e4c054430 con 0x7f9e540b1910 2023-06-23T18:36:35.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.853+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2051031354 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9e540c7570 con 0x7f9e540b1910 2023-06-23T18:36:35.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.853+0000 7f9e50ff9700 1 -- 172.21.15.79:0/2051031354 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9e4c057a50 con 0x7f9e540b1910 2023-06-23T18:36:35.859 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:35.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.855+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2051031354 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9e4006ace0 msgr2=0x7f9e4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:35.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.855+0000 7f9e5a6dd700 1 --2- 172.21.15.79:0/2051031354 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9e4006ace0 0x7f9e4006d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9e5415d120 tx=0x7f9e48008040 comp rx=0 tx=0).stop 2023-06-23T18:36:35.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.855+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2051031354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e540b1910 msgr2=0x7f9e540c88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:35.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.855+0000 7f9e5a6dd700 1 --2- 172.21.15.79:0/2051031354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e540b1910 0x7f9e540c88a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9e54071270 tx=0x7f9e4c00ee70 comp rx=0 tx=0).stop 2023-06-23T18:36:35.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.855+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2051031354 shutdown_connections 2023-06-23T18:36:35.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.855+0000 7f9e5a6dd700 1 --2- 172.21.15.79:0/2051031354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e540b1910 0x7f9e540c88a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.855+0000 7f9e5a6dd700 1 --2- 172.21.15.79:0/2051031354 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9e4006ace0 0x7f9e4006d190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.856+0000 7f9e5a6dd700 1 --2- 172.21.15.79:0/2051031354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e3c004850 0x7f9e540b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.856+0000 7f9e5a6dd700 1 --2- 172.21.15.79:0/2051031354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e540c6e00 0x7f9e540c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:35.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.856+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2051031354 >> 172.21.15.79:0/2051031354 conn(0x7f9e5406afa0 msgr2=0x7f9e54158290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:35.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.856+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2051031354 shutdown_connections 2023-06-23T18:36:35.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.856+0000 7f9e5a6dd700 1 -- 172.21.15.79:0/2051031354 wait complete. 2023-06-23T18:36:35.872 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:35.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.992+0000 7f5aed915700 1 Processor -- start 2023-06-23T18:36:35.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.992+0000 7f5aed915700 1 -- start start 2023-06-23T18:36:35.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5aed915700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ae80cfd10 0x7f5ae80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5aed915700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5ae80b1a80 0x7f5ae80d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:35.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5aed915700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae80d1830 con 0x7f5ae80b81c0 2023-06-23T18:36:35.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5aed915700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae80cf330 con 0x7f5ae80b1a80 2023-06-23T18:36:35.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5aed915700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae80d1270 con 0x7f5ae80cfd10 2023-06-23T18:36:35.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5ae77fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5ae80b1a80 0x7f5ae80d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5ae77fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5ae80b1a80 0x7f5ae80d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40676/0 (socket says 172.21.15.79:40676) 2023-06-23T18:36:36.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5ae77fe700 1 -- 172.21.15.79:0/2011151789 learned_addr learned my addr 172.21.15.79:0/2011151789 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:36.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5ae67fc700 1 --2- 172.21.15.79:0/2011151789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ae80cfd10 0x7f5ae80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5ae77fe700 1 -- 172.21.15.79:0/2011151789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ae80cfd10 msgr2=0x7f5ae80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:36.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5ae77fe700 1 --2- 172.21.15.79:0/2011151789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ae80cfd10 0x7f5ae80d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5ae77fe700 1 -- 172.21.15.79:0/2011151789 >> v1:172.21.15.155:6789/0 conn(0x7f5ae80b81c0 legacy=0x7f5ae80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:36.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5ae77fe700 1 -- 172.21.15.79:0/2011151789 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae80d1550 con 0x7f5ae80b1a80 2023-06-23T18:36:36.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5ae77fe700 1 --2- 172.21.15.79:0/2011151789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ae80b1a80 0x7f5ae80d28c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009e80 tx=0x7f5ae000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec3b4c9ca176aaa server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:36.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5ae5ffb700 1 -- 172.21.15.79:0/2011151789 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000e070 con 0x7f5ae80b1a80 2023-06-23T18:36:36.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5ae5ffb700 1 -- 172.21.15.79:0/2011151789 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae0003a20 con 0x7f5ae80b1a80 2023-06-23T18:36:36.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.993+0000 7f5ae5ffb700 1 -- 172.21.15.79:0/2011151789 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0003dd0 con 0x7f5ae80b1a80 2023-06-23T18:36:36.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.994+0000 7f5aed915700 1 -- 172.21.15.79:0/2011151789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ae80b1a80 msgr2=0x7f5ae80d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.994+0000 7f5aed915700 1 --2- 172.21.15.79:0/2011151789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ae80b1a80 0x7f5ae80d28c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009e80 tx=0x7f5ae000b330 comp rx=0 tx=0).stop 2023-06-23T18:36:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.994+0000 7f5aed915700 1 -- 172.21.15.79:0/2011151789 shutdown_connections 2023-06-23T18:36:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.994+0000 7f5aed915700 1 --2- 172.21.15.79:0/2011151789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ae80b1a80 0x7f5ae80d28c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.994+0000 7f5aed915700 1 --2- 172.21.15.79:0/2011151789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ae80cfd10 0x7f5ae80d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.994+0000 7f5aed915700 1 -- 172.21.15.79:0/2011151789 >> 172.21.15.79:0/2011151789 conn(0x7f5ae806b2b0 msgr2=0x7f5ae806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.994+0000 7f5aed915700 1 -- 172.21.15.79:0/2011151789 shutdown_connections 2023-06-23T18:36:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.994+0000 7f5aed915700 1 -- 172.21.15.79:0/2011151789 wait complete. 2023-06-23T18:36:36.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.995+0000 7f5aed915700 1 Processor -- start 2023-06-23T18:36:36.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.995+0000 7f5aed915700 1 -- start start 2023-06-23T18:36:36.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5aed915700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ae80b1a80 0x7f5ae814ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5aed915700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ae80b81c0 0x7f5ae814a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5aed915700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae80cfd10 0x7f5ae80caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5aed915700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae815d910 con 0x7f5ae80b1a80 2023-06-23T18:36:36.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5aed915700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae814b0b0 con 0x7f5ae80cfd10 2023-06-23T18:36:36.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5aed915700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae815ca20 con 0x7f5ae80b81c0 2023-06-23T18:36:36.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5ae6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ae80b1a80 0x7f5ae814ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5ae6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ae80b1a80 0x7f5ae814ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40678/0 (socket says 172.21.15.79:40678) 2023-06-23T18:36:36.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5ae6ffd700 1 -- 172.21.15.79:0/2622561210 learned_addr learned my addr 172.21.15.79:0/2622561210 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:36.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5ae77fe700 1 --2- 172.21.15.79:0/2622561210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae80cfd10 0x7f5ae80caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5ae67fc700 1 --2- 172.21.15.79:0/2622561210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ae80b81c0 0x7f5ae814a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5ae6ffd700 1 -- 172.21.15.79:0/2622561210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ae80b81c0 msgr2=0x7f5ae814a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5ae6ffd700 1 --2- 172.21.15.79:0/2622561210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ae80b81c0 0x7f5ae814a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5ae6ffd700 1 -- 172.21.15.79:0/2622561210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae80cfd10 msgr2=0x7f5ae80caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.996+0000 7f5ae6ffd700 1 --2- 172.21.15.79:0/2622561210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae80cfd10 0x7f5ae80caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.997+0000 7f5ae6ffd700 1 -- 172.21.15.79:0/2622561210 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae0009b30 con 0x7f5ae80b1a80 2023-06-23T18:36:36.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.997+0000 7f5ae6ffd700 1 --2- 172.21.15.79:0/2622561210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ae80b1a80 0x7f5ae814ab70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00c4d0 tx=0x7f5adc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:36.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.997+0000 7f5acffff700 1 -- 172.21.15.79:0/2622561210 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc011790 con 0x7f5ae80b1a80 2023-06-23T18:36:36.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.997+0000 7f5acffff700 1 -- 172.21.15.79:0/2622561210 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc0118f0 con 0x7f5ae80b1a80 2023-06-23T18:36:36.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.997+0000 7f5aed915700 1 -- 172.21.15.79:0/2622561210 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae815cd00 con 0x7f5ae80b1a80 2023-06-23T18:36:36.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.997+0000 7f5acffff700 1 -- 172.21.15.79:0/2622561210 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc011ca0 con 0x7f5ae80b1a80 2023-06-23T18:36:36.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.997+0000 7f5aed915700 1 -- 172.21.15.79:0/2622561210 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae815d2c0 con 0x7f5ae80b1a80 2023-06-23T18:36:36.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.998+0000 7f5aed915700 1 -- 172.21.15.79:0/2622561210 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae804e1d0 con 0x7f5ae80b1a80 2023-06-23T18:36:36.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.998+0000 7f5acffff700 1 -- 172.21.15.79:0/2622561210 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5adc01c460 con 0x7f5ae80b1a80 2023-06-23T18:36:36.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.999+0000 7f5acffff700 1 --2- 172.21.15.79:0/2622561210 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5ad406ace0 0x7f5ad406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.999+0000 7f5acffff700 1 -- 172.21.15.79:0/2622561210 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5adc089350 con 0x7f5ae80b1a80 2023-06-23T18:36:36.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.999+0000 7f5ae67fc700 1 --2- 172.21.15.79:0/2622561210 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5ad406ace0 0x7f5ad406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:35.999+0000 7f5ae67fc700 1 --2- 172.21.15.79:0/2622561210 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5ad406ace0 0x7f5ad406d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80b8af0 tx=0x7f5ad8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:36.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.003+0000 7f5acffff700 1 -- 172.21.15.79:0/2622561210 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5adc054370 con 0x7f5ae80b1a80 2023-06-23T18:36:36.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.189+0000 7f5aed915700 1 -- 172.21.15.79:0/2622561210 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5ae80b4f30 con 0x7f5ae80b1a80 2023-06-23T18:36:36.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.189+0000 7f5acffff700 1 -- 172.21.15.79:0/2622561210 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5adc057990 con 0x7f5ae80b1a80 2023-06-23T18:36:36.195 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:36.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.191+0000 7f5aed915700 1 -- 172.21.15.79:0/2622561210 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5ad406ace0 msgr2=0x7f5ad406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.191+0000 7f5aed915700 1 --2- 172.21.15.79:0/2622561210 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5ad406ace0 0x7f5ad406d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80b8af0 tx=0x7f5ad8008040 comp rx=0 tx=0).stop 2023-06-23T18:36:36.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.191+0000 7f5aed915700 1 -- 172.21.15.79:0/2622561210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ae80b1a80 msgr2=0x7f5ae814ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.191+0000 7f5aed915700 1 --2- 172.21.15.79:0/2622561210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ae80b1a80 0x7f5ae814ab70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00c4d0 tx=0x7f5adc00c890 comp rx=0 tx=0).stop 2023-06-23T18:36:36.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.191+0000 7f5aed915700 1 -- 172.21.15.79:0/2622561210 shutdown_connections 2023-06-23T18:36:36.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.191+0000 7f5aed915700 1 --2- 172.21.15.79:0/2622561210 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5ad406ace0 0x7f5ad406d190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.191+0000 7f5aed915700 1 --2- 172.21.15.79:0/2622561210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae80cfd10 0x7f5ae80caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.191+0000 7f5aed915700 1 --2- 172.21.15.79:0/2622561210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ae80b1a80 0x7f5ae814ab70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.191+0000 7f5aed915700 1 --2- 172.21.15.79:0/2622561210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ae80b81c0 0x7f5ae814a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.191+0000 7f5aed915700 1 -- 172.21.15.79:0/2622561210 >> 172.21.15.79:0/2622561210 conn(0x7f5ae806b2b0 msgr2=0x7f5ae80732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:36.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.192+0000 7f5aed915700 1 -- 172.21.15.79:0/2622561210 shutdown_connections 2023-06-23T18:36:36.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.192+0000 7f5aed915700 1 -- 172.21.15.79:0/2622561210 wait complete. 2023-06-23T18:36:36.208 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:36.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.326+0000 7ff3405d6700 1 Processor -- start 2023-06-23T18:36:36.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.326+0000 7ff3405d6700 1 -- start start 2023-06-23T18:36:36.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.327+0000 7ff3405d6700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff3381569c0 0x7ff3381608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.327+0000 7ff3405d6700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3380cdc90 con 0x7ff3380cd440 2023-06-23T18:36:36.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.327+0000 7ff3405d6700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33815d0f0 con 0x7ff33806a5a0 2023-06-23T18:36:36.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.327+0000 7ff3405d6700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3380cf570 con 0x7ff3381569c0 2023-06-23T18:36:36.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.327+0000 7ff33db71700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff3380cd440 0x7ff338069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43728/0 (socket says 172.21.15.79:43728) 2023-06-23T18:36:36.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.327+0000 7ff33db71700 1 -- 172.21.15.79:0/3435562741 learned_addr learned my addr 172.21.15.79:0/3435562741 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:36.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 -- 172.21.15.79:0/3435562741 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503107717 0 0) 0x7ff328001110 con 0x7ff3380cd440 2023-06-23T18:36:36.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33e372700 1 --2- 172.21.15.79:0/3435562741 >> v2:172.21.15.155:3300/0 conn(0x7ff3381569c0 0x7ff3381608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 -- 172.21.15.79:0/3435562741 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3240035b0 con 0x7ff3380cd440 2023-06-23T18:36:36.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 -- 172.21.15.79:0/3435562741 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 841419982 0 0) 0x7ff3240035b0 con 0x7ff3380cd440 2023-06-23T18:36:36.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 -- 172.21.15.79:0/3435562741 >> v2:172.21.15.155:3300/0 conn(0x7ff3381569c0 msgr2=0x7ff3381608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:36.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 --2- 172.21.15.79:0/3435562741 >> v2:172.21.15.155:3300/0 conn(0x7ff3381569c0 0x7ff3381608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 -- 172.21.15.79:0/3435562741 >> v1:172.21.15.155:6789/0 conn(0x7ff33806a5a0 legacy=0x7ff3380b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 -- 172.21.15.79:0/3435562741 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff338156d90 con 0x7ff3380cd440 2023-06-23T18:36:36.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 -- 172.21.15.79:0/3435562741 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7ff328001800 con 0x7ff3380cd440 2023-06-23T18:36:36.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 -- 172.21.15.79:0/3435562741 >> v1:172.21.15.79:6789/0 conn(0x7ff3380cd440 legacy=0x7ff338069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 --2- 172.21.15.79:0/3435562741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff324003a00 0x7ff324003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 --2- 172.21.15.79:0/3435562741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240043f0 0x7ff3240048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 --2- 172.21.15.79:0/3435562741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3240050b0 0x7ff32400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 -- 172.21.15.79:0/3435562741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32400c000 con 0x7ff3240043f0 2023-06-23T18:36:36.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 -- 172.21.15.79:0/3435562741 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32400c140 con 0x7ff324003a00 2023-06-23T18:36:36.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33d370700 1 -- 172.21.15.79:0/3435562741 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32400c300 con 0x7ff3240050b0 2023-06-23T18:36:36.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33db71700 1 --2- 172.21.15.79:0/3435562741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240043f0 0x7ff3240048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33e372700 1 --2- 172.21.15.79:0/3435562741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff324003a00 0x7ff324003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33eb73700 1 --2- 172.21.15.79:0/3435562741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3240050b0 0x7ff32400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33db71700 1 -- 172.21.15.79:0/3435562741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3240050b0 msgr2=0x7ff32400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:36.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33db71700 1 --2- 172.21.15.79:0/3435562741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3240050b0 0x7ff32400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33db71700 1 -- 172.21.15.79:0/3435562741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff324003a00 msgr2=0x7ff324003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:36.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33db71700 1 --2- 172.21.15.79:0/3435562741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff324003a00 0x7ff324003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33db71700 1 -- 172.21.15.79:0/3435562741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3240035b0 con 0x7ff3240043f0 2023-06-23T18:36:36.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.328+0000 7ff33eb73700 1 -- 172.21.15.79:0/3435562741 reap_dead start 2023-06-23T18:36:36.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.329+0000 7ff33db71700 1 --2- 172.21.15.79:0/3435562741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240043f0 0x7ff3240048b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff32800c240 tx=0x7ff324005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961f3e19e1d1a70c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:36.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.329+0000 7ff33d370700 1 -- 172.21.15.79:0/3435562741 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32800cda0 con 0x7ff3240043f0 2023-06-23T18:36:36.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.329+0000 7ff33d370700 1 -- 172.21.15.79:0/3435562741 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff328005960 con 0x7ff3240043f0 2023-06-23T18:36:36.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.329+0000 7ff33d370700 1 -- 172.21.15.79:0/3435562741 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff328005d10 con 0x7ff3240043f0 2023-06-23T18:36:36.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.329+0000 7ff3405d6700 1 -- 172.21.15.79:0/3435562741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240043f0 msgr2=0x7ff3240048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.329+0000 7ff3405d6700 1 --2- 172.21.15.79:0/3435562741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240043f0 0x7ff3240048b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff32800c240 tx=0x7ff324005880 comp rx=0 tx=0).stop 2023-06-23T18:36:36.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.329+0000 7ff3405d6700 1 -- 172.21.15.79:0/3435562741 shutdown_connections 2023-06-23T18:36:36.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.329+0000 7ff3405d6700 1 --2- 172.21.15.79:0/3435562741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240043f0 0x7ff3240048b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.329+0000 7ff3405d6700 1 -- 172.21.15.79:0/3435562741 >> 172.21.15.79:0/3435562741 conn(0x7ff33806afa0 msgr2=0x7ff33806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:36.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.329+0000 7ff3405d6700 1 -- 172.21.15.79:0/3435562741 shutdown_connections 2023-06-23T18:36:36.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.329+0000 7ff3405d6700 1 -- 172.21.15.79:0/3435562741 wait complete. 2023-06-23T18:36:36.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.330+0000 7ff3405d6700 1 Processor -- start 2023-06-23T18:36:36.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.330+0000 7ff3405d6700 1 -- start start 2023-06-23T18:36:36.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.331+0000 7ff3405d6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240043f0 0x7ff3380b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.331+0000 7ff3405d6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3380adf20 0x7ff3381530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.331+0000 7ff3405d6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3380c91f0 0x7ff33815ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.331+0000 7ff3405d6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff338153610 con 0x7ff3240043f0 2023-06-23T18:36:36.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.331+0000 7ff3405d6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3380c6c50 con 0x7ff3380c91f0 2023-06-23T18:36:36.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.331+0000 7ff3405d6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3380ae4e0 con 0x7ff3380adf20 2023-06-23T18:36:36.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.331+0000 7ff33e372700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240043f0 0x7ff3380b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.331+0000 7ff33e372700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240043f0 0x7ff3380b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40698/0 (socket says 172.21.15.79:40698) 2023-06-23T18:36:36.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.331+0000 7ff33e372700 1 -- 172.21.15.79:0/3843381376 learned_addr learned my addr 172.21.15.79:0/3843381376 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:36.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.331+0000 7ff33eb73700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3380c91f0 0x7ff33815ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.331+0000 7ff33db71700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3380adf20 0x7ff3381530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.332+0000 7ff33e372700 1 -- 172.21.15.79:0/3843381376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3380adf20 msgr2=0x7ff3381530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.332+0000 7ff33e372700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3380adf20 0x7ff3381530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.332+0000 7ff33e372700 1 -- 172.21.15.79:0/3843381376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3380c91f0 msgr2=0x7ff33815ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.332+0000 7ff33e372700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3380c91f0 0x7ff33815ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.332+0000 7ff33e372700 1 -- 172.21.15.79:0/3843381376 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3380c6f10 con 0x7ff3240043f0 2023-06-23T18:36:36.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.332+0000 7ff33db71700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3380adf20 0x7ff3381530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:36.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.332+0000 7ff33eb73700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3380c91f0 0x7ff33815ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:36.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.332+0000 7ff33e372700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240043f0 0x7ff3380b26a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff334011aa0 tx=0x7ff334011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:36.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.332+0000 7ff32effd700 1 -- 172.21.15.79:0/3843381376 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33400f980 con 0x7ff3240043f0 2023-06-23T18:36:36.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.332+0000 7ff32effd700 1 -- 172.21.15.79:0/3843381376 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33400fb00 con 0x7ff3240043f0 2023-06-23T18:36:36.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.332+0000 7ff3405d6700 1 -- 172.21.15.79:0/3843381376 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3380c71f0 con 0x7ff3240043f0 2023-06-23T18:36:36.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.332+0000 7ff32effd700 1 -- 172.21.15.79:0/3843381376 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33400feb0 con 0x7ff3240043f0 2023-06-23T18:36:36.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.332+0000 7ff3405d6700 1 -- 172.21.15.79:0/3843381376 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33815e300 con 0x7ff3240043f0 2023-06-23T18:36:36.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.333+0000 7ff3405d6700 1 -- 172.21.15.79:0/3843381376 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3380672c0 con 0x7ff3240043f0 2023-06-23T18:36:36.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.333+0000 7ff32effd700 1 -- 172.21.15.79:0/3843381376 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff334014440 con 0x7ff3240043f0 2023-06-23T18:36:36.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.334+0000 7ff32effd700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff31c06ace0 0x7ff31c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.334+0000 7ff33db71700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff31c06ace0 0x7ff31c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.334+0000 7ff32effd700 1 -- 172.21.15.79:0/3843381376 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff334089c40 con 0x7ff3240043f0 2023-06-23T18:36:36.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.334+0000 7ff33db71700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff31c06ace0 0x7ff31c06d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff328001ac0 tx=0x7ff328007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:36.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.338+0000 7ff32effd700 1 -- 172.21.15.79:0/3843381376 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff3340588d0 con 0x7ff3240043f0 2023-06-23T18:36:36.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.522+0000 7ff3405d6700 1 -- 172.21.15.79:0/3843381376 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff3380ccb70 con 0x7ff3240043f0 2023-06-23T18:36:36.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.522+0000 7ff32effd700 1 -- 172.21.15.79:0/3843381376 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff334058460 con 0x7ff3240043f0 2023-06-23T18:36:36.528 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:36.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.524+0000 7ff3405d6700 1 -- 172.21.15.79:0/3843381376 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff31c06ace0 msgr2=0x7ff31c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.524+0000 7ff3405d6700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff31c06ace0 0x7ff31c06d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff328001ac0 tx=0x7ff328007000 comp rx=0 tx=0).stop 2023-06-23T18:36:36.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.524+0000 7ff3405d6700 1 -- 172.21.15.79:0/3843381376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240043f0 msgr2=0x7ff3380b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.524+0000 7ff3405d6700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240043f0 0x7ff3380b26a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff334011aa0 tx=0x7ff334011e60 comp rx=0 tx=0).stop 2023-06-23T18:36:36.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.524+0000 7ff3405d6700 1 -- 172.21.15.79:0/3843381376 shutdown_connections 2023-06-23T18:36:36.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.524+0000 7ff3405d6700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff31c06ace0 0x7ff31c06d190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.524+0000 7ff3405d6700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3380c91f0 0x7ff33815ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.524+0000 7ff3405d6700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240043f0 0x7ff3380b26a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.524+0000 7ff3405d6700 1 --2- 172.21.15.79:0/3843381376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3380adf20 0x7ff3381530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.524+0000 7ff3405d6700 1 -- 172.21.15.79:0/3843381376 >> 172.21.15.79:0/3843381376 conn(0x7ff33806afa0 msgr2=0x7ff33815f380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:36.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.525+0000 7ff3405d6700 1 -- 172.21.15.79:0/3843381376 shutdown_connections 2023-06-23T18:36:36.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.525+0000 7ff3405d6700 1 -- 172.21.15.79:0/3843381376 wait complete. 2023-06-23T18:36:36.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.659+0000 7f2272690700 1 Processor -- start 2023-06-23T18:36:36.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.659+0000 7f2272690700 1 -- start start 2023-06-23T18:36:36.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.660+0000 7f2272690700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c1569c0 0x7f226c1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.660+0000 7f2272690700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f226c0cdc90 con 0x7f226c0cd440 2023-06-23T18:36:36.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.660+0000 7f2272690700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f226c15d0f0 con 0x7f226c06a5a0 2023-06-23T18:36:36.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.660+0000 7f2272690700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0cf570 con 0x7f226c1569c0 2023-06-23T18:36:36.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.660+0000 7f226b7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f226c0cd440 0x7f226c069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43748/0 (socket says 172.21.15.79:43748) 2023-06-23T18:36:36.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.660+0000 7f226b7fe700 1 -- 172.21.15.79:0/69323372 learned_addr learned my addr 172.21.15.79:0/69323372 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:36.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.660+0000 7f226bfff700 1 --2- 172.21.15.79:0/69323372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c1569c0 0x7f226c1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.661+0000 7f226affd700 1 -- 172.21.15.79:0/69323372 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3835055562 0 0) 0x7f225c001110 con 0x7f226c0cd440 2023-06-23T18:36:36.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.661+0000 7f226affd700 1 -- 172.21.15.79:0/69323372 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22540035b0 con 0x7f226c0cd440 2023-06-23T18:36:36.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.661+0000 7f226affd700 1 -- 172.21.15.79:0/69323372 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603007839 0 0) 0x7f226c15d0f0 con 0x7f226c06a5a0 2023-06-23T18:36:36.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.661+0000 7f226affd700 1 -- 172.21.15.79:0/69323372 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f225c001110 con 0x7f226c06a5a0 2023-06-23T18:36:36.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.661+0000 7f226bfff700 1 -- 172.21.15.79:0/69323372 >> v1:172.21.15.155:6789/0 conn(0x7f226c06a5a0 legacy=0x7f226c0b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.661+0000 7f226bfff700 1 -- 172.21.15.79:0/69323372 >> v1:172.21.15.79:6789/0 conn(0x7f226c0cd440 legacy=0x7f226c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.661+0000 7f226bfff700 1 -- 172.21.15.79:0/69323372 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c156d90 con 0x7f226c1569c0 2023-06-23T18:36:36.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.661+0000 7f226bfff700 1 --2- 172.21.15.79:0/69323372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c1569c0 0x7f226c1608b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2260009870 tx=0x7f2260009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e68fbb7a32c76e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:36.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.661+0000 7f226affd700 1 -- 172.21.15.79:0/69323372 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2260011830 con 0x7f226c1569c0 2023-06-23T18:36:36.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.661+0000 7f226affd700 1 -- 172.21.15.79:0/69323372 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2260011990 con 0x7f226c1569c0 2023-06-23T18:36:36.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.661+0000 7f226affd700 1 -- 172.21.15.79:0/69323372 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2260011d00 con 0x7f226c1569c0 2023-06-23T18:36:36.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.662+0000 7f2272690700 1 -- 172.21.15.79:0/69323372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c1569c0 msgr2=0x7f226c1608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.662+0000 7f2272690700 1 --2- 172.21.15.79:0/69323372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c1569c0 0x7f226c1608b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2260009870 tx=0x7f2260009b80 comp rx=0 tx=0).stop 2023-06-23T18:36:36.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.662+0000 7f2272690700 1 -- 172.21.15.79:0/69323372 shutdown_connections 2023-06-23T18:36:36.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.662+0000 7f2272690700 1 --2- 172.21.15.79:0/69323372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c1569c0 0x7f226c1608b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.662+0000 7f2272690700 1 -- 172.21.15.79:0/69323372 >> 172.21.15.79:0/69323372 conn(0x7f226c06afa0 msgr2=0x7f226c06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:36.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.662+0000 7f2272690700 1 -- 172.21.15.79:0/69323372 shutdown_connections 2023-06-23T18:36:36.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.662+0000 7f2272690700 1 -- 172.21.15.79:0/69323372 wait complete. 2023-06-23T18:36:36.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.663+0000 7f2272690700 1 Processor -- start 2023-06-23T18:36:36.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.663+0000 7f2272690700 1 -- start start 2023-06-23T18:36:36.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f2272690700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f226c06a5a0 0x7f226c148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f2272690700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226c0cd440 0x7f226c0c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f2272690700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c1569c0 0x7f226c0c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f2272690700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c15d580 con 0x7f226c06a5a0 2023-06-23T18:36:36.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f2272690700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c15d860 con 0x7f226c0cd440 2023-06-23T18:36:36.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f2272690700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c15db40 con 0x7f226c1569c0 2023-06-23T18:36:36.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f226bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f226c06a5a0 0x7f226c148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f226bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f226c06a5a0 0x7f226c148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40712/0 (socket says 172.21.15.79:40712) 2023-06-23T18:36:36.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f226bfff700 1 -- 172.21.15.79:0/2319498860 learned_addr learned my addr 172.21.15.79:0/2319498860 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:36.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f226b7fe700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226c0cd440 0x7f226c0c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f2270c2d700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c1569c0 0x7f226c0c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f226bfff700 1 -- 172.21.15.79:0/2319498860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c1569c0 msgr2=0x7f226c0c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f226bfff700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c1569c0 0x7f226c0c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f226bfff700 1 -- 172.21.15.79:0/2319498860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226c0cd440 msgr2=0x7f226c0c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f226bfff700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226c0cd440 0x7f226c0c5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f226bfff700 1 -- 172.21.15.79:0/2319498860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22600094a0 con 0x7f226c06a5a0 2023-06-23T18:36:36.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f2270c2d700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c1569c0 0x7f226c0c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:36.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f226bfff700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f226c06a5a0 0x7f226c148ae0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2260009bc0 tx=0x7f2260004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:36.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f2268ff9700 1 -- 172.21.15.79:0/2319498860 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22600045c0 con 0x7f226c06a5a0 2023-06-23T18:36:36.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f2268ff9700 1 -- 172.21.15.79:0/2319498860 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226001e070 con 0x7f226c06a5a0 2023-06-23T18:36:36.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f2272690700 1 -- 172.21.15.79:0/2319498860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f226c15de20 con 0x7f226c06a5a0 2023-06-23T18:36:36.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.664+0000 7f2268ff9700 1 -- 172.21.15.79:0/2319498860 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2260003900 con 0x7f226c06a5a0 2023-06-23T18:36:36.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.665+0000 7f2272690700 1 -- 172.21.15.79:0/2319498860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226c0b31e0 con 0x7f226c06a5a0 2023-06-23T18:36:36.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.665+0000 7f2272690700 1 -- 172.21.15.79:0/2319498860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226c15e340 con 0x7f226c06a5a0 2023-06-23T18:36:36.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.666+0000 7f2268ff9700 1 -- 172.21.15.79:0/2319498860 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2260003a60 con 0x7f226c06a5a0 2023-06-23T18:36:36.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.666+0000 7f2268ff9700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f225806ace0 0x7f225806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:36.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.666+0000 7f226b7fe700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f225806ace0 0x7f225806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:36.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.666+0000 7f2268ff9700 1 -- 172.21.15.79:0/2319498860 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f226008a390 con 0x7f226c06a5a0 2023-06-23T18:36:36.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.667+0000 7f226b7fe700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f225806ace0 0x7f225806d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f225c000f60 tx=0x7f225c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:36.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.671+0000 7f2268ff9700 1 -- 172.21.15.79:0/2319498860 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f226c15e340 con 0x7f226c06a5a0 2023-06-23T18:36:36.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.856+0000 7f2272690700 1 -- 172.21.15.79:0/2319498860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f226c15e340 con 0x7f226c06a5a0 2023-06-23T18:36:36.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.857+0000 7f2268ff9700 1 -- 172.21.15.79:0/2319498860 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f226c15e340 con 0x7f226c06a5a0 2023-06-23T18:36:36.862 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:36.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.859+0000 7f2272690700 1 -- 172.21.15.79:0/2319498860 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f225806ace0 msgr2=0x7f225806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.859+0000 7f2272690700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f225806ace0 0x7f225806d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f225c000f60 tx=0x7f225c009040 comp rx=0 tx=0).stop 2023-06-23T18:36:36.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.859+0000 7f2272690700 1 -- 172.21.15.79:0/2319498860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f226c06a5a0 msgr2=0x7f226c148ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:36.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.859+0000 7f2272690700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f226c06a5a0 0x7f226c148ae0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2260009bc0 tx=0x7f2260004750 comp rx=0 tx=0).stop 2023-06-23T18:36:36.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.859+0000 7f2272690700 1 -- 172.21.15.79:0/2319498860 shutdown_connections 2023-06-23T18:36:36.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.859+0000 7f2272690700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c1569c0 0x7f226c0c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.859+0000 7f2272690700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f226c06a5a0 0x7f226c148ae0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.859+0000 7f2272690700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f225806ace0 0x7f225806d190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.859+0000 7f2272690700 1 --2- 172.21.15.79:0/2319498860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226c0cd440 0x7f226c0c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:36.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.859+0000 7f2272690700 1 -- 172.21.15.79:0/2319498860 >> 172.21.15.79:0/2319498860 conn(0x7f226c06afa0 msgr2=0x7f226c15f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:36.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.859+0000 7f2272690700 1 -- 172.21.15.79:0/2319498860 shutdown_connections 2023-06-23T18:36:36.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.859+0000 7f2272690700 1 -- 172.21.15.79:0/2319498860 wait complete. 2023-06-23T18:36:36.877 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:36:36.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.993+0000 7f728ad81700 1 Processor -- start 2023-06-23T18:36:36.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.993+0000 7f728ad81700 1 -- start start 2023-06-23T18:36:36.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.993+0000 7f728ad81700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72840cec00 0x7f72840b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.993+0000 7f728ad81700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f728415a750 0x7f72840b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.993+0000 7f728ad81700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72840b1d40 con 0x7f728415a380 2023-06-23T18:36:37.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.993+0000 7f728ad81700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72840b1a90 con 0x7f728415a750 2023-06-23T18:36:37.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.994+0000 7f728ad81700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72840b2020 con 0x7f72840cec00 2023-06-23T18:36:37.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.994+0000 7f728931e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f728415a750 0x7f72840b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.994+0000 7f728931e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f728415a750 0x7f72840b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40724/0 (socket says 172.21.15.79:40724) 2023-06-23T18:36:37.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.994+0000 7f728931e700 1 -- 172.21.15.79:0/2331180184 learned_addr learned my addr 172.21.15.79:0/2331180184 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:37.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.994+0000 7f728931e700 1 -- 172.21.15.79:0/2331180184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72840cec00 msgr2=0x7f72840b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:37.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.994+0000 7f7283fff700 1 --2- 172.21.15.79:0/2331180184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72840cec00 0x7f72840b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.994+0000 7f728931e700 1 --2- 172.21.15.79:0/2331180184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72840cec00 0x7f72840b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.994+0000 7f728931e700 1 -- 172.21.15.79:0/2331180184 >> v1:172.21.15.155:6789/0 conn(0x7f728415a380 legacy=0x7f72840ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:37.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.994+0000 7f728931e700 1 -- 172.21.15.79:0/2331180184 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72840ad650 con 0x7f728415a750 2023-06-23T18:36:37.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.994+0000 7f7283fff700 1 --2- 172.21.15.79:0/2331180184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72840cec00 0x7f72840b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:37.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.994+0000 7f728931e700 1 --2- 172.21.15.79:0/2331180184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728415a750 0x7f72840b0540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f727c009e50 tx=0x7f727c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0c577db857d554 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:37.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.994+0000 7f72837fe700 1 -- 172.21.15.79:0/2331180184 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c00e070 con 0x7f728415a750 2023-06-23T18:36:37.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.994+0000 7f72837fe700 1 -- 172.21.15.79:0/2331180184 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c003a20 con 0x7f728415a750 2023-06-23T18:36:37.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.994+0000 7f72837fe700 1 -- 172.21.15.79:0/2331180184 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c003dd0 con 0x7f728415a750 2023-06-23T18:36:37.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.995+0000 7f728ad81700 1 -- 172.21.15.79:0/2331180184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728415a750 msgr2=0x7f72840b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.995+0000 7f728ad81700 1 --2- 172.21.15.79:0/2331180184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728415a750 0x7f72840b0540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f727c009e50 tx=0x7f727c00b330 comp rx=0 tx=0).stop 2023-06-23T18:36:37.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.995+0000 7f728ad81700 1 -- 172.21.15.79:0/2331180184 shutdown_connections 2023-06-23T18:36:37.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.995+0000 7f728ad81700 1 --2- 172.21.15.79:0/2331180184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728415a750 0x7f72840b0540 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.995+0000 7f728ad81700 1 --2- 172.21.15.79:0/2331180184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72840cec00 0x7f72840b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.995+0000 7f728ad81700 1 -- 172.21.15.79:0/2331180184 >> 172.21.15.79:0/2331180184 conn(0x7f7284089fc0 msgr2=0x7f7284099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:37.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.995+0000 7f728ad81700 1 -- 172.21.15.79:0/2331180184 shutdown_connections 2023-06-23T18:36:37.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.995+0000 7f728ad81700 1 -- 172.21.15.79:0/2331180184 wait complete. 2023-06-23T18:36:37.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.996+0000 7f728ad81700 1 Processor -- start 2023-06-23T18:36:37.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.996+0000 7f728ad81700 1 -- start start 2023-06-23T18:36:37.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f728ad81700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72840cec00 0x7f7284151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f728ad81700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f728415a380 0x7f72840c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f728ad81700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728415a750 0x7f72840c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f728ad81700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7284151590 con 0x7f728415a750 2023-06-23T18:36:37.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f728ad81700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72840b4e10 con 0x7f72840cec00 2023-06-23T18:36:37.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f728ad81700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72840c65d0 con 0x7f728415a380 2023-06-23T18:36:37.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f728931e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728415a750 0x7f72840c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f728931e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728415a750 0x7f72840c56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40730/0 (socket says 172.21.15.79:40730) 2023-06-23T18:36:37.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f728931e700 1 -- 172.21.15.79:0/961046896 learned_addr learned my addr 172.21.15.79:0/961046896 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:37.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f7288b1d700 1 --2- 172.21.15.79:0/961046896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72840cec00 0x7f7284151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f7283fff700 1 --2- 172.21.15.79:0/961046896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f728415a380 0x7f72840c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f728931e700 1 -- 172.21.15.79:0/961046896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f728415a380 msgr2=0x7f72840c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f728931e700 1 --2- 172.21.15.79:0/961046896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f728415a380 0x7f72840c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f728931e700 1 -- 172.21.15.79:0/961046896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72840cec00 msgr2=0x7f7284151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f728931e700 1 --2- 172.21.15.79:0/961046896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72840cec00 0x7f7284151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.997+0000 7f728931e700 1 -- 172.21.15.79:0/961046896 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c009b30 con 0x7f728415a750 2023-06-23T18:36:37.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.998+0000 7f728931e700 1 --2- 172.21.15.79:0/961046896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728415a750 0x7f72840c56f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f727c004000 tx=0x7f727c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:37.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.998+0000 7f72817fa700 1 -- 172.21.15.79:0/961046896 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c004440 con 0x7f728415a750 2023-06-23T18:36:37.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.998+0000 7f72817fa700 1 -- 172.21.15.79:0/961046896 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c0045c0 con 0x7f728415a750 2023-06-23T18:36:37.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.998+0000 7f728ad81700 1 -- 172.21.15.79:0/961046896 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72840c68b0 con 0x7f728415a750 2023-06-23T18:36:37.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.998+0000 7f72817fa700 1 -- 172.21.15.79:0/961046896 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c004970 con 0x7f728415a750 2023-06-23T18:36:37.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.998+0000 7f728ad81700 1 -- 172.21.15.79:0/961046896 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72840c6e70 con 0x7f728415a750 2023-06-23T18:36:37.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.999+0000 7f728ad81700 1 -- 172.21.15.79:0/961046896 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728404e240 con 0x7f728415a750 2023-06-23T18:36:37.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.999+0000 7f72817fa700 1 -- 172.21.15.79:0/961046896 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f727c012430 con 0x7f728415a750 2023-06-23T18:36:37.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.999+0000 7f72817fa700 1 --2- 172.21.15.79:0/961046896 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f726806ace0 0x7f726806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:36.999+0000 7f72817fa700 1 -- 172.21.15.79:0/961046896 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f727c08a830 con 0x7f728415a750 2023-06-23T18:36:37.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.000+0000 7f7288b1d700 1 --2- 172.21.15.79:0/961046896 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f726806ace0 0x7f726806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.000+0000 7f7288b1d700 1 --2- 172.21.15.79:0/961046896 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f726806ace0 0x7f726806d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f728406e960 tx=0x7f7278007f10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:37.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.004+0000 7f72817fa700 1 -- 172.21.15.79:0/961046896 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f727c059400 con 0x7f728415a750 2023-06-23T18:36:37.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.212+0000 7f728ad81700 1 -- 172.21.15.79:0/961046896 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f728406a950 con 0x7f728415a750 2023-06-23T18:36:37.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.213+0000 7f72817fa700 1 -- 172.21.15.79:0/961046896 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f727c058f90 con 0x7f728415a750 2023-06-23T18:36:37.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.214+0000 7f728ad81700 1 -- 172.21.15.79:0/961046896 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f726806ace0 msgr2=0x7f726806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.214+0000 7f728ad81700 1 --2- 172.21.15.79:0/961046896 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f726806ace0 0x7f726806d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f728406e960 tx=0x7f7278007f10 comp rx=0 tx=0).stop 2023-06-23T18:36:37.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.215+0000 7f728ad81700 1 -- 172.21.15.79:0/961046896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728415a750 msgr2=0x7f72840c56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.215+0000 7f728ad81700 1 --2- 172.21.15.79:0/961046896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728415a750 0x7f72840c56f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f727c004000 tx=0x7f727c004ad0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.215+0000 7f728ad81700 1 -- 172.21.15.79:0/961046896 shutdown_connections 2023-06-23T18:36:37.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.215+0000 7f728ad81700 1 --2- 172.21.15.79:0/961046896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f728415a750 0x7f72840c56f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.215+0000 7f728ad81700 1 --2- 172.21.15.79:0/961046896 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f726806ace0 0x7f726806d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.215+0000 7f728ad81700 1 --2- 172.21.15.79:0/961046896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72840cec00 0x7f7284151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.215+0000 7f728ad81700 1 --2- 172.21.15.79:0/961046896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f728415a380 0x7f72840c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.215+0000 7f728ad81700 1 -- 172.21.15.79:0/961046896 >> 172.21.15.79:0/961046896 conn(0x7f7284089fc0 msgr2=0x7f72840b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:37.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.215+0000 7f728ad81700 1 -- 172.21.15.79:0/961046896 shutdown_connections 2023-06-23T18:36:37.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.215+0000 7f728ad81700 1 -- 172.21.15.79:0/961046896 wait complete. 2023-06-23T18:36:37.222 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-23T18:36:37.231 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:36:37.232 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:36:37.248 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:37.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.403+0000 7fc2cdd5e700 1 Processor -- start 2023-06-23T18:36:37.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.403+0000 7fc2cdd5e700 1 -- start start 2023-06-23T18:36:37.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.404+0000 7fc2cdd5e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc2c815fa60 0x7fc2c80ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.404+0000 7fc2cdd5e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c8160de0 con 0x7fc2c81609f0 2023-06-23T18:36:37.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.404+0000 7fc2cdd5e700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c815fe30 con 0x7fc2c80b17d0 2023-06-23T18:36:37.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.404+0000 7fc2cdd5e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c806ce10 con 0x7fc2c815fa60 2023-06-23T18:36:37.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.404+0000 7fc2c77fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc2c815fa60 0x7fc2c80ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.404+0000 7fc2c77fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc2c815fa60 0x7fc2c80ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40734/0 (socket says 172.21.15.79:40734) 2023-06-23T18:36:37.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.404+0000 7fc2c77fe700 1 -- 172.21.15.79:0/3908248979 learned_addr learned my addr 172.21.15.79:0/3908248979 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:37.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.404+0000 7fc2c7fff700 1 --1- 172.21.15.79:0/3908248979 >> v1:172.21.15.79:6789/0 conn(0x7fc2c81609f0 0x7fc2c806afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43780/0 (socket says 172.21.15.79:43780) 2023-06-23T18:36:37.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.404+0000 7fc2c77fe700 1 -- 172.21.15.79:0/3908248979 >> v1:172.21.15.155:6789/0 conn(0x7fc2c80b17d0 legacy=0x7fc2c81602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:37.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.404+0000 7fc2c77fe700 1 -- 172.21.15.79:0/3908248979 >> v1:172.21.15.79:6789/0 conn(0x7fc2c81609f0 legacy=0x7fc2c806afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.404+0000 7fc2c77fe700 1 -- 172.21.15.79:0/3908248979 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c80b0a10 con 0x7fc2c815fa60 2023-06-23T18:36:37.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.404+0000 7fc2c67fc700 1 -- 172.21.15.79:0/3908248979 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210471146 0 0) 0x7fc2c8160de0 con 0x7fc2c81609f0 2023-06-23T18:36:37.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.404+0000 7fc2c77fe700 1 --2- 172.21.15.79:0/3908248979 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c815fa60 0x7fc2c80ce4f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0006fd0 tx=0x7fc2c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d526bcb5efc719bd server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:37.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.404+0000 7fc2c67fc700 1 -- 172.21.15.79:0/3908248979 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0005e30 con 0x7fc2c815fa60 2023-06-23T18:36:37.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.405+0000 7fc2c67fc700 1 -- 172.21.15.79:0/3908248979 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c0004030 con 0x7fc2c815fa60 2023-06-23T18:36:37.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.405+0000 7fc2c67fc700 1 -- 172.21.15.79:0/3908248979 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00043e0 con 0x7fc2c815fa60 2023-06-23T18:36:37.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.405+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/3908248979 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c815fa60 msgr2=0x7fc2c80ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.405+0000 7fc2cdd5e700 1 --2- 172.21.15.79:0/3908248979 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c815fa60 0x7fc2c80ce4f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0006fd0 tx=0x7fc2c0009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.405+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/3908248979 shutdown_connections 2023-06-23T18:36:37.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.405+0000 7fc2cdd5e700 1 --2- 172.21.15.79:0/3908248979 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c815fa60 0x7fc2c80ce4f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.405+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/3908248979 >> 172.21.15.79:0/3908248979 conn(0x7fc2c8089fc0 msgr2=0x7fc2c8099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:37.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.405+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/3908248979 shutdown_connections 2023-06-23T18:36:37.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.405+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/3908248979 wait complete. 2023-06-23T18:36:37.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.406+0000 7fc2cdd5e700 1 Processor -- start 2023-06-23T18:36:37.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.406+0000 7fc2cdd5e700 1 -- start start 2023-06-23T18:36:37.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.407+0000 7fc2cdd5e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c80b17d0 0x7fc2c81566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.407+0000 7fc2cdd5e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2c815fa60 0x7fc2c80b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.407+0000 7fc2cdd5e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c81609f0 0x7fc2c815c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.407+0000 7fc2cdd5e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80c5a90 con 0x7fc2c80b17d0 2023-06-23T18:36:37.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.407+0000 7fc2cdd5e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80c5d70 con 0x7fc2c81609f0 2023-06-23T18:36:37.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.407+0000 7fc2cdd5e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80c6050 con 0x7fc2c815fa60 2023-06-23T18:36:37.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.407+0000 7fc2c77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c80b17d0 0x7fc2c81566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.407+0000 7fc2c77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c80b17d0 0x7fc2c81566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40740/0 (socket says 172.21.15.79:40740) 2023-06-23T18:36:37.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.407+0000 7fc2c77fe700 1 -- 172.21.15.79:0/4141295057 learned_addr learned my addr 172.21.15.79:0/4141295057 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:37.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.408+0000 7fc2c7fff700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c81609f0 0x7fc2c815c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.408+0000 7fc2c6ffd700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2c815fa60 0x7fc2c80b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.408+0000 7fc2c77fe700 1 -- 172.21.15.79:0/4141295057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2c815fa60 msgr2=0x7fc2c80b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.408+0000 7fc2c77fe700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2c815fa60 0x7fc2c80b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.408+0000 7fc2c77fe700 1 -- 172.21.15.79:0/4141295057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c81609f0 msgr2=0x7fc2c815c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.408+0000 7fc2c77fe700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c81609f0 0x7fc2c815c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.408+0000 7fc2c77fe700 1 -- 172.21.15.79:0/4141295057 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c0009500 con 0x7fc2c80b17d0 2023-06-23T18:36:37.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.408+0000 7fc2c7fff700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c81609f0 0x7fc2c815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:37.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.408+0000 7fc2c77fe700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c80b17d0 0x7fc2c81566f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009130 tx=0x7fc2c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:37.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.408+0000 7fc2bffff700 1 -- 172.21.15.79:0/4141295057 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0011620 con 0x7fc2c80b17d0 2023-06-23T18:36:37.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.408+0000 7fc2bffff700 1 -- 172.21.15.79:0/4141295057 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c0011780 con 0x7fc2c80b17d0 2023-06-23T18:36:37.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.408+0000 7fc2bffff700 1 -- 172.21.15.79:0/4141295057 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0011b30 con 0x7fc2c80b17d0 2023-06-23T18:36:37.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.408+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/4141295057 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c80c6330 con 0x7fc2c80b17d0 2023-06-23T18:36:37.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.408+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/4141295057 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c81e77b0 con 0x7fc2c80b17d0 2023-06-23T18:36:37.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.409+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/4141295057 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c806c500 con 0x7fc2c80b17d0 2023-06-23T18:36:37.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.410+0000 7fc2bffff700 1 -- 172.21.15.79:0/4141295057 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc2c002f480 con 0x7fc2c80b17d0 2023-06-23T18:36:37.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.410+0000 7fc2bffff700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc2a806ace0 0x7fc2a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.410+0000 7fc2bffff700 1 -- 172.21.15.79:0/4141295057 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc2c008ac50 con 0x7fc2c80b17d0 2023-06-23T18:36:37.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.410+0000 7fc2c6ffd700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc2a806ace0 0x7fc2a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.411+0000 7fc2c6ffd700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc2a806ace0 0x7fc2a806d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc2c815e520 tx=0x7fc2b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:37.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.414+0000 7fc2bffff700 1 -- 172.21.15.79:0/4141295057 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc2c008ae60 con 0x7fc2c80b17d0 2023-06-23T18:36:37.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.602+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/4141295057 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2c80c5bd0 con 0x7fc2c80b17d0 2023-06-23T18:36:37.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.603+0000 7fc2bffff700 1 -- 172.21.15.79:0/4141295057 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7fc2c80c5bd0 con 0x7fc2c80b17d0 2023-06-23T18:36:37.608 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:37.609 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:37.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.606+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/4141295057 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc2a806ace0 msgr2=0x7fc2a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.606+0000 7fc2cdd5e700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc2a806ace0 0x7fc2a806d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc2c815e520 tx=0x7fc2b0009040 comp rx=0 tx=0).stop 2023-06-23T18:36:37.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.606+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/4141295057 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c80b17d0 msgr2=0x7fc2c81566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.606+0000 7fc2cdd5e700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c80b17d0 0x7fc2c81566f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009130 tx=0x7fc2c0004750 comp rx=0 tx=0).stop 2023-06-23T18:36:37.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.606+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/4141295057 shutdown_connections 2023-06-23T18:36:37.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.606+0000 7fc2cdd5e700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc2a806ace0 0x7fc2a806d190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.606+0000 7fc2cdd5e700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c81609f0 0x7fc2c815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.606+0000 7fc2cdd5e700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c80b17d0 0x7fc2c81566f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.606+0000 7fc2cdd5e700 1 --2- 172.21.15.79:0/4141295057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2c815fa60 0x7fc2c80b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.606+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/4141295057 >> 172.21.15.79:0/4141295057 conn(0x7fc2c8089fc0 msgr2=0x7fc2c815efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:37.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.606+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/4141295057 shutdown_connections 2023-06-23T18:36:37.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.606+0000 7fc2cdd5e700 1 -- 172.21.15.79:0/4141295057 wait complete. 2023-06-23T18:36:37.623 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:37.623 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:37.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.742+0000 7fb480855700 1 Processor -- start 2023-06-23T18:36:37.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.742+0000 7fb480855700 1 -- start start 2023-06-23T18:36:37.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.743+0000 7fb480855700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb47c156ae0 0x7fb47c15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.743+0000 7fb480855700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb47c0cc060 0x7fb47c0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.743+0000 7fb480855700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb47c0cc9f0 con 0x7fb47c071710 2023-06-23T18:36:37.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.743+0000 7fb480855700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb47c15daf0 con 0x7fb47c156ae0 2023-06-23T18:36:37.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.743+0000 7fb480855700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c15a560 con 0x7fb47c0cc060 2023-06-23T18:36:37.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.743+0000 7fb47a59c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb47c156ae0 0x7fb47c15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.743+0000 7fb47a59c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb47c156ae0 0x7fb47c15d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40748/0 (socket says 172.21.15.79:40748) 2023-06-23T18:36:37.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.743+0000 7fb47a59c700 1 -- 172.21.15.79:0/2044642638 learned_addr learned my addr 172.21.15.79:0/2044642638 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:37.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.743+0000 7fb479d9b700 1 --2- 172.21.15.79:0/2044642638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb47c0cc060 0x7fb47c0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.743+0000 7fb47a59c700 1 -- 172.21.15.79:0/2044642638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb47c0cc060 msgr2=0x7fb47c0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:37.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.743+0000 7fb47a59c700 1 --2- 172.21.15.79:0/2044642638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb47c0cc060 0x7fb47c0cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.743+0000 7fb47a59c700 1 -- 172.21.15.79:0/2044642638 >> v1:172.21.15.155:6789/0 conn(0x7fb47c071710 legacy=0x7fb47c071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.743+0000 7fb47a59c700 1 -- 172.21.15.79:0/2044642638 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47c159950 con 0x7fb47c156ae0 2023-06-23T18:36:37.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.744+0000 7fb47a59c700 1 --2- 172.21.15.79:0/2044642638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb47c156ae0 0x7fb47c15d5b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb4700098f0 tx=0x7fb470009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a047361845469c88 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:37.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.744+0000 7fb47959a700 1 -- 172.21.15.79:0/2044642638 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47000b8a0 con 0x7fb47c156ae0 2023-06-23T18:36:37.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.744+0000 7fb47959a700 1 -- 172.21.15.79:0/2044642638 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47000ba00 con 0x7fb47c156ae0 2023-06-23T18:36:37.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.744+0000 7fb47959a700 1 -- 172.21.15.79:0/2044642638 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47000bd90 con 0x7fb47c156ae0 2023-06-23T18:36:37.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.744+0000 7fb480855700 1 -- 172.21.15.79:0/2044642638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb47c156ae0 msgr2=0x7fb47c15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.744+0000 7fb480855700 1 --2- 172.21.15.79:0/2044642638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb47c156ae0 0x7fb47c15d5b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb4700098f0 tx=0x7fb470009c00 comp rx=0 tx=0).stop 2023-06-23T18:36:37.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.744+0000 7fb480855700 1 -- 172.21.15.79:0/2044642638 shutdown_connections 2023-06-23T18:36:37.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.744+0000 7fb480855700 1 --2- 172.21.15.79:0/2044642638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb47c156ae0 0x7fb47c15d5b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.744+0000 7fb480855700 1 --2- 172.21.15.79:0/2044642638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb47c0cc060 0x7fb47c0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.744+0000 7fb480855700 1 -- 172.21.15.79:0/2044642638 >> 172.21.15.79:0/2044642638 conn(0x7fb47c06afa0 msgr2=0x7fb47c06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:37.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.744+0000 7fb480855700 1 -- 172.21.15.79:0/2044642638 shutdown_connections 2023-06-23T18:36:37.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.744+0000 7fb480855700 1 -- 172.21.15.79:0/2044642638 wait complete. 2023-06-23T18:36:37.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.746+0000 7fb480855700 1 Processor -- start 2023-06-23T18:36:37.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.746+0000 7fb480855700 1 -- start start 2023-06-23T18:36:37.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.746+0000 7fb480855700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb47c071710 0x7fb47c14ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.746+0000 7fb480855700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb47c0cc060 0x7fb47c14e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.746+0000 7fb480855700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb47c156ae0 0x7fb47c149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.746+0000 7fb480855700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c0af020 con 0x7fb47c156ae0 2023-06-23T18:36:37.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.746+0000 7fb480855700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c0b5160 con 0x7fb47c071710 2023-06-23T18:36:37.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.746+0000 7fb480855700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c0b5440 con 0x7fb47c0cc060 2023-06-23T18:36:37.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb47ad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb47c156ae0 0x7fb47c149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb47a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb47c071710 0x7fb47c14ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb47ad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb47c156ae0 0x7fb47c149e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40758/0 (socket says 172.21.15.79:40758) 2023-06-23T18:36:37.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb47ad9d700 1 -- 172.21.15.79:0/1462904898 learned_addr learned my addr 172.21.15.79:0/1462904898 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:37.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb479d9b700 1 --2- 172.21.15.79:0/1462904898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb47c0cc060 0x7fb47c14e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb47ad9d700 1 -- 172.21.15.79:0/1462904898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb47c0cc060 msgr2=0x7fb47c14e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb47ad9d700 1 --2- 172.21.15.79:0/1462904898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb47c0cc060 0x7fb47c14e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb47ad9d700 1 -- 172.21.15.79:0/1462904898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb47c071710 msgr2=0x7fb47c14ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb47ad9d700 1 --2- 172.21.15.79:0/1462904898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb47c071710 0x7fb47c14ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb47ad9d700 1 -- 172.21.15.79:0/1462904898 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb470009550 con 0x7fb47c156ae0 2023-06-23T18:36:37.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb479d9b700 1 --2- 172.21.15.79:0/1462904898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb47c0cc060 0x7fb47c14e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:37.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb47ad9d700 1 --2- 172.21.15.79:0/1462904898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb47c156ae0 0x7fb47c149e60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb47400cc50 tx=0x7fb47400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:37.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb46affd700 1 -- 172.21.15.79:0/1462904898 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474003c00 con 0x7fb47c156ae0 2023-06-23T18:36:37.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb46affd700 1 -- 172.21.15.79:0/1462904898 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb474003d60 con 0x7fb47c156ae0 2023-06-23T18:36:37.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb46affd700 1 -- 172.21.15.79:0/1462904898 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47400ab60 con 0x7fb47c156ae0 2023-06-23T18:36:37.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb480855700 1 -- 172.21.15.79:0/1462904898 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47c0ae9d0 con 0x7fb47c156ae0 2023-06-23T18:36:37.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.747+0000 7fb480855700 1 -- 172.21.15.79:0/1462904898 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47c0b26a0 con 0x7fb47c156ae0 2023-06-23T18:36:37.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.748+0000 7fb480855700 1 -- 172.21.15.79:0/1462904898 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47c0af190 con 0x7fb47c156ae0 2023-06-23T18:36:37.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.749+0000 7fb46affd700 1 -- 172.21.15.79:0/1462904898 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb47400acc0 con 0x7fb47c156ae0 2023-06-23T18:36:37.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.749+0000 7fb46affd700 1 --2- 172.21.15.79:0/1462904898 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb45c06ace0 0x7fb45c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:37.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.749+0000 7fb46affd700 1 -- 172.21.15.79:0/1462904898 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4740892e0 con 0x7fb47c156ae0 2023-06-23T18:36:37.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.749+0000 7fb47a59c700 1 --2- 172.21.15.79:0/1462904898 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb45c06ace0 0x7fb45c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:37.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.749+0000 7fb47a59c700 1 --2- 172.21.15.79:0/1462904898 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb45c06ace0 0x7fb45c06d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb470014040 tx=0x7fb470008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:37.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.753+0000 7fb46affd700 1 -- 172.21.15.79:0/1462904898 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb474054390 con 0x7fb47c156ae0 2023-06-23T18:36:37.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.939+0000 7fb480855700 1 -- 172.21.15.79:0/1462904898 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb47c0cc9f0 con 0x7fb47c156ae0 2023-06-23T18:36:37.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.939+0000 7fb46affd700 1 -- 172.21.15.79:0/1462904898 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb47c0cc9f0 con 0x7fb47c156ae0 2023-06-23T18:36:37.944 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:37.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.941+0000 7fb480855700 1 -- 172.21.15.79:0/1462904898 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb45c06ace0 msgr2=0x7fb45c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.941+0000 7fb480855700 1 --2- 172.21.15.79:0/1462904898 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb45c06ace0 0x7fb45c06d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb470014040 tx=0x7fb470008030 comp rx=0 tx=0).stop 2023-06-23T18:36:37.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.941+0000 7fb480855700 1 -- 172.21.15.79:0/1462904898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb47c156ae0 msgr2=0x7fb47c149e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:37.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.941+0000 7fb480855700 1 --2- 172.21.15.79:0/1462904898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb47c156ae0 0x7fb47c149e60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb47400cc50 tx=0x7fb47400cf60 comp rx=0 tx=0).stop 2023-06-23T18:36:37.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.941+0000 7fb480855700 1 -- 172.21.15.79:0/1462904898 shutdown_connections 2023-06-23T18:36:37.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.941+0000 7fb480855700 1 --2- 172.21.15.79:0/1462904898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb47c156ae0 0x7fb47c149e60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.941+0000 7fb480855700 1 --2- 172.21.15.79:0/1462904898 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fb45c06ace0 0x7fb45c06d190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.941+0000 7fb480855700 1 --2- 172.21.15.79:0/1462904898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb47c071710 0x7fb47c14ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.941+0000 7fb480855700 1 --2- 172.21.15.79:0/1462904898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb47c0cc060 0x7fb47c14e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:37.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.941+0000 7fb480855700 1 -- 172.21.15.79:0/1462904898 >> 172.21.15.79:0/1462904898 conn(0x7fb47c06afa0 msgr2=0x7fb47c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:37.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.941+0000 7fb480855700 1 -- 172.21.15.79:0/1462904898 shutdown_connections 2023-06-23T18:36:37.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:37.941+0000 7fb480855700 1 -- 172.21.15.79:0/1462904898 wait complete. 2023-06-23T18:36:37.958 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:38.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.075+0000 7f98220bc700 1 Processor -- start 2023-06-23T18:36:38.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.075+0000 7f98220bc700 1 -- start start 2023-06-23T18:36:38.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.076+0000 7f98220bc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f981c0cfd10 0x7f981c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.076+0000 7f98220bc700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f981c0b1a80 0x7f981c0d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.076+0000 7f98220bc700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981c0d1830 con 0x7f981c0b81c0 2023-06-23T18:36:38.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.076+0000 7f98220bc700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c0cf330 con 0x7f981c0b1a80 2023-06-23T18:36:38.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.076+0000 7f98220bc700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0d1270 con 0x7f981c0cfd10 2023-06-23T18:36:38.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.076+0000 7f981affd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f981c0cfd10 0x7f981c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.076+0000 7f981bfff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f981c0b1a80 0x7f981c0d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.076+0000 7f981affd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f981c0cfd10 0x7f981c0d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:33534/0 (socket says 172.21.15.79:33534) 2023-06-23T18:36:38.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.076+0000 7f981affd700 1 -- 172.21.15.79:0/1203623348 learned_addr learned my addr 172.21.15.79:0/1203623348 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:38.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.076+0000 7f981affd700 1 -- 172.21.15.79:0/1203623348 >> v2:172.21.15.155:3300/0 conn(0x7f981c0b1a80 msgr2=0x7f981c0d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:38.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.076+0000 7f981affd700 1 --2- 172.21.15.79:0/1203623348 >> v2:172.21.15.155:3300/0 conn(0x7f981c0b1a80 0x7f981c0d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.076+0000 7f981affd700 1 -- 172.21.15.79:0/1203623348 >> v1:172.21.15.155:6789/0 conn(0x7f981c0b81c0 legacy=0x7f981c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.076+0000 7f981affd700 1 -- 172.21.15.79:0/1203623348 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c0d1550 con 0x7f981c0cfd10 2023-06-23T18:36:38.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.077+0000 7f981affd700 1 --2- 172.21.15.79:0/1203623348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f981c0cfd10 0x7f981c0d2380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f980c009920 tx=0x7f980c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0ae578ee93690d2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:38.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.077+0000 7f981a7fc700 1 -- 172.21.15.79:0/1203623348 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c011830 con 0x7f981c0cfd10 2023-06-23T18:36:38.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.077+0000 7f981a7fc700 1 -- 172.21.15.79:0/1203623348 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c011990 con 0x7f981c0cfd10 2023-06-23T18:36:38.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.077+0000 7f981a7fc700 1 -- 172.21.15.79:0/1203623348 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c011d20 con 0x7f981c0cfd10 2023-06-23T18:36:38.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.077+0000 7f98220bc700 1 -- 172.21.15.79:0/1203623348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f981c0cfd10 msgr2=0x7f981c0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.077+0000 7f98220bc700 1 --2- 172.21.15.79:0/1203623348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f981c0cfd10 0x7f981c0d2380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f980c009920 tx=0x7f980c009c30 comp rx=0 tx=0).stop 2023-06-23T18:36:38.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.077+0000 7f98220bc700 1 -- 172.21.15.79:0/1203623348 shutdown_connections 2023-06-23T18:36:38.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.077+0000 7f98220bc700 1 --2- 172.21.15.79:0/1203623348 >> v2:172.21.15.155:3300/0 conn(0x7f981c0b1a80 0x7f981c0d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.077+0000 7f98220bc700 1 --2- 172.21.15.79:0/1203623348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f981c0cfd10 0x7f981c0d2380 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.077+0000 7f98220bc700 1 -- 172.21.15.79:0/1203623348 >> 172.21.15.79:0/1203623348 conn(0x7f981c06b2b0 msgr2=0x7f981c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:38.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.077+0000 7f98220bc700 1 -- 172.21.15.79:0/1203623348 shutdown_connections 2023-06-23T18:36:38.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.077+0000 7f98220bc700 1 -- 172.21.15.79:0/1203623348 wait complete. 2023-06-23T18:36:38.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.079+0000 7f98220bc700 1 Processor -- start 2023-06-23T18:36:38.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.079+0000 7f98220bc700 1 -- start start 2023-06-23T18:36:38.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.079+0000 7f98220bc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f981c0b1a80 0x7f981c0cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.079+0000 7f98220bc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f981c0b81c0 0x7f981c0b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.079+0000 7f98220bc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f981c0cfd10 0x7f981c0b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.079+0000 7f98220bc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0cab40 con 0x7f981c0b81c0 2023-06-23T18:36:38.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f98220bc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0cae20 con 0x7f981c0b1a80 2023-06-23T18:36:38.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f98220bc700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0cb100 con 0x7f981c0cfd10 2023-06-23T18:36:38.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f981affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f981c0b81c0 0x7f981c0b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f981affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f981c0b81c0 0x7f981c0b6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40770/0 (socket says 172.21.15.79:40770) 2023-06-23T18:36:38.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f981affd700 1 -- 172.21.15.79:0/190116924 learned_addr learned my addr 172.21.15.79:0/190116924 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:38.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f981b7fe700 1 --2- 172.21.15.79:0/190116924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f981c0b1a80 0x7f981c0cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f981bfff700 1 --2- 172.21.15.79:0/190116924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f981c0cfd10 0x7f981c0b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f981affd700 1 -- 172.21.15.79:0/190116924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f981c0cfd10 msgr2=0x7f981c0b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f981affd700 1 --2- 172.21.15.79:0/190116924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f981c0cfd10 0x7f981c0b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f981affd700 1 -- 172.21.15.79:0/190116924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f981c0b1a80 msgr2=0x7f981c0cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f981affd700 1 --2- 172.21.15.79:0/190116924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f981c0b1a80 0x7f981c0cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f981affd700 1 -- 172.21.15.79:0/190116924 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c009550 con 0x7f981c0b81c0 2023-06-23T18:36:38.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f981affd700 1 --2- 172.21.15.79:0/190116924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f981c0b81c0 0x7f981c0b6ee0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f980c014040 tx=0x7f980c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:38.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f9803fff700 1 -- 172.21.15.79:0/190116924 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c004510 con 0x7f981c0b81c0 2023-06-23T18:36:38.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f9803fff700 1 -- 172.21.15.79:0/190116924 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c01e070 con 0x7f981c0b81c0 2023-06-23T18:36:38.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f98220bc700 1 -- 172.21.15.79:0/190116924 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c0cb3e0 con 0x7f981c0b81c0 2023-06-23T18:36:38.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f9803fff700 1 -- 172.21.15.79:0/190116924 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c0038e0 con 0x7f981c0b81c0 2023-06-23T18:36:38.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.080+0000 7f98220bc700 1 -- 172.21.15.79:0/190116924 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c0ca6d0 con 0x7f981c0b81c0 2023-06-23T18:36:38.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.081+0000 7f98220bc700 1 -- 172.21.15.79:0/190116924 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c0cb240 con 0x7f981c0b81c0 2023-06-23T18:36:38.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.082+0000 7f9803fff700 1 -- 172.21.15.79:0/190116924 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f980c003a40 con 0x7f981c0b81c0 2023-06-23T18:36:38.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.082+0000 7f9803fff700 1 --2- 172.21.15.79:0/190116924 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f980806ace0 0x7f980806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.082+0000 7f9803fff700 1 -- 172.21.15.79:0/190116924 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f980c08a2c0 con 0x7f981c0b81c0 2023-06-23T18:36:38.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.082+0000 7f981b7fe700 1 --2- 172.21.15.79:0/190116924 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f980806ace0 0x7f980806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.082+0000 7f981b7fe700 1 --2- 172.21.15.79:0/190116924 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f980806ace0 0x7f980806d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f981c0cd210 tx=0x7f9810009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:38.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.086+0000 7f9803fff700 1 -- 172.21.15.79:0/190116924 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f981c0cb240 con 0x7f981c0b81c0 2023-06-23T18:36:38.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.272+0000 7f98220bc700 1 -- 172.21.15.79:0/190116924 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f981c0cb240 con 0x7f981c0b81c0 2023-06-23T18:36:38.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.272+0000 7f9803fff700 1 -- 172.21.15.79:0/190116924 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f981c0cb240 con 0x7f981c0b81c0 2023-06-23T18:36:38.277 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:38.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.274+0000 7f98220bc700 1 -- 172.21.15.79:0/190116924 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f980806ace0 msgr2=0x7f980806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.274+0000 7f98220bc700 1 --2- 172.21.15.79:0/190116924 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f980806ace0 0x7f980806d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f981c0cd210 tx=0x7f9810009040 comp rx=0 tx=0).stop 2023-06-23T18:36:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.274+0000 7f98220bc700 1 -- 172.21.15.79:0/190116924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f981c0b81c0 msgr2=0x7f981c0b6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.274+0000 7f98220bc700 1 --2- 172.21.15.79:0/190116924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f981c0b81c0 0x7f981c0b6ee0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f980c014040 tx=0x7f980c004750 comp rx=0 tx=0).stop 2023-06-23T18:36:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.274+0000 7f98220bc700 1 -- 172.21.15.79:0/190116924 shutdown_connections 2023-06-23T18:36:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.274+0000 7f98220bc700 1 --2- 172.21.15.79:0/190116924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f981c0cfd10 0x7f981c0b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.274+0000 7f98220bc700 1 --2- 172.21.15.79:0/190116924 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f980806ace0 0x7f980806d190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.274+0000 7f98220bc700 1 --2- 172.21.15.79:0/190116924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f981c0b1a80 0x7f981c0cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.274+0000 7f98220bc700 1 --2- 172.21.15.79:0/190116924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f981c0b81c0 0x7f981c0b6ee0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.274+0000 7f98220bc700 1 -- 172.21.15.79:0/190116924 >> 172.21.15.79:0/190116924 conn(0x7f981c06b2b0 msgr2=0x7f981c0d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.274+0000 7f98220bc700 1 -- 172.21.15.79:0/190116924 shutdown_connections 2023-06-23T18:36:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.274+0000 7f98220bc700 1 -- 172.21.15.79:0/190116924 wait complete. 2023-06-23T18:36:38.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:38.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.409+0000 7f9479650700 1 Processor -- start 2023-06-23T18:36:38.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.409+0000 7f9479650700 1 -- start start 2023-06-23T18:36:38.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9479650700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f947415c420 0x7f9474070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9479650700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94740ce730 0x7f94740b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9479650700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947406bf00 con 0x7f94740caf40 2023-06-23T18:36:38.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9479650700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f947406b9b0 con 0x7f947415c420 2023-06-23T18:36:38.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9479650700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947406c710 con 0x7f94740ce730 2023-06-23T18:36:38.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9472ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f947415c420 0x7f9474070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9472ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f947415c420 0x7f9474070af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40776/0 (socket says 172.21.15.79:40776) 2023-06-23T18:36:38.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9472ffd700 1 -- 172.21.15.79:0/3853666143 learned_addr learned my addr 172.21.15.79:0/3853666143 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:38.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9472ffd700 1 -- 172.21.15.79:0/3853666143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94740ce730 msgr2=0x7f94740b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:38.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f94737fe700 1 --2- 172.21.15.79:0/3853666143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94740ce730 0x7f94740b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9472ffd700 1 --2- 172.21.15.79:0/3853666143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94740ce730 0x7f94740b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9472ffd700 1 -- 172.21.15.79:0/3853666143 >> v1:172.21.15.155:6789/0 conn(0x7f94740caf40 legacy=0x7f947406c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:38.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9472ffd700 1 -- 172.21.15.79:0/3853666143 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94740cb3b0 con 0x7f947415c420 2023-06-23T18:36:38.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9472ffd700 1 --2- 172.21.15.79:0/3853666143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f947415c420 0x7f9474070af0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f94680098f0 tx=0x7f9468009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4653cd74b65d4fe9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:38.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9471ffb700 1 -- 172.21.15.79:0/3853666143 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468004230 con 0x7f947415c420 2023-06-23T18:36:38.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.410+0000 7f9471ffb700 1 -- 172.21.15.79:0/3853666143 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468004390 con 0x7f947415c420 2023-06-23T18:36:38.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.411+0000 7f9471ffb700 1 -- 172.21.15.79:0/3853666143 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680036a0 con 0x7f947415c420 2023-06-23T18:36:38.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.411+0000 7f9479650700 1 -- 172.21.15.79:0/3853666143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f947415c420 msgr2=0x7f9474070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.411+0000 7f9479650700 1 --2- 172.21.15.79:0/3853666143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f947415c420 0x7f9474070af0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f94680098f0 tx=0x7f9468009c00 comp rx=0 tx=0).stop 2023-06-23T18:36:38.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.411+0000 7f9479650700 1 -- 172.21.15.79:0/3853666143 shutdown_connections 2023-06-23T18:36:38.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.411+0000 7f9479650700 1 --2- 172.21.15.79:0/3853666143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f947415c420 0x7f9474070af0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.411+0000 7f9479650700 1 --2- 172.21.15.79:0/3853666143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94740ce730 0x7f94740b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.411+0000 7f9479650700 1 -- 172.21.15.79:0/3853666143 >> 172.21.15.79:0/3853666143 conn(0x7f947406e540 msgr2=0x7f947406e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:38.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.411+0000 7f9479650700 1 -- 172.21.15.79:0/3853666143 shutdown_connections 2023-06-23T18:36:38.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.411+0000 7f9479650700 1 -- 172.21.15.79:0/3853666143 wait complete. 2023-06-23T18:36:38.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.412+0000 7f9479650700 1 Processor -- start 2023-06-23T18:36:38.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.412+0000 7f9479650700 1 -- start start 2023-06-23T18:36:38.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.413+0000 7f9479650700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94740caf40 0x7f947414dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.413+0000 7f9479650700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94740ce730 0x7f94740c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.413+0000 7f9479650700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f947415c420 0x7f94740b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.413+0000 7f9479650700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947414e2a0 con 0x7f947415c420 2023-06-23T18:36:38.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.413+0000 7f9479650700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947415d5e0 con 0x7f94740caf40 2023-06-23T18:36:38.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.413+0000 7f9479650700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947415d8c0 con 0x7f94740ce730 2023-06-23T18:36:38.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.413+0000 7f94737fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f947415c420 0x7f94740b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.413+0000 7f94737fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f947415c420 0x7f94740b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40784/0 (socket says 172.21.15.79:40784) 2023-06-23T18:36:38.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.413+0000 7f94737fe700 1 -- 172.21.15.79:0/648427217 learned_addr learned my addr 172.21.15.79:0/648427217 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:38.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.413+0000 7f9472ffd700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94740caf40 0x7f947414dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.414+0000 7f94727fc700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94740ce730 0x7f94740c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.414+0000 7f94737fe700 1 -- 172.21.15.79:0/648427217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94740ce730 msgr2=0x7f94740c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.414+0000 7f94737fe700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94740ce730 0x7f94740c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.414+0000 7f94737fe700 1 -- 172.21.15.79:0/648427217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94740caf40 msgr2=0x7f947414dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.414+0000 7f94737fe700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94740caf40 0x7f947414dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.414+0000 7f94737fe700 1 -- 172.21.15.79:0/648427217 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9468009550 con 0x7f947415c420 2023-06-23T18:36:38.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.414+0000 7f9472ffd700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94740caf40 0x7f947414dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:38.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.414+0000 7f94737fe700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f947415c420 0x7f94740b3740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f946c00cd70 tx=0x7f946c00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:38.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.414+0000 7f945b7fe700 1 -- 172.21.15.79:0/648427217 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c003a20 con 0x7f947415c420 2023-06-23T18:36:38.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.414+0000 7f945b7fe700 1 -- 172.21.15.79:0/648427217 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c003b80 con 0x7f947415c420 2023-06-23T18:36:38.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.414+0000 7f9479650700 1 -- 172.21.15.79:0/648427217 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947415dba0 con 0x7f947415c420 2023-06-23T18:36:38.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.414+0000 7f945b7fe700 1 -- 172.21.15.79:0/648427217 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c004020 con 0x7f947415c420 2023-06-23T18:36:38.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.414+0000 7f9479650700 1 -- 172.21.15.79:0/648427217 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947415e160 con 0x7f947415c420 2023-06-23T18:36:38.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.415+0000 7f9479650700 1 -- 172.21.15.79:0/648427217 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947415e380 con 0x7f947415c420 2023-06-23T18:36:38.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.415+0000 7f945b7fe700 1 -- 172.21.15.79:0/648427217 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f946c004180 con 0x7f947415c420 2023-06-23T18:36:38.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.416+0000 7f945b7fe700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f946006ace0 0x7f946006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.416+0000 7f9472ffd700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f946006ace0 0x7f946006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.416+0000 7f945b7fe700 1 -- 172.21.15.79:0/648427217 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f946c012070 con 0x7f947415c420 2023-06-23T18:36:38.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.416+0000 7f9472ffd700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f946006ace0 0x7f946006d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9468009c40 tx=0x7f9468008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:38.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.420+0000 7f945b7fe700 1 -- 172.21.15.79:0/648427217 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f946c054da0 con 0x7f947415c420 2023-06-23T18:36:38.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.607+0000 7f9479650700 1 -- 172.21.15.79:0/648427217 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94740d0440 con 0x7f947415c420 2023-06-23T18:36:38.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.607+0000 7f945b7fe700 1 -- 172.21.15.79:0/648427217 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f946c0583c0 con 0x7f947415c420 2023-06-23T18:36:38.613 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:38.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.609+0000 7f9479650700 1 -- 172.21.15.79:0/648427217 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f946006ace0 msgr2=0x7f946006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.609+0000 7f9479650700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f946006ace0 0x7f946006d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9468009c40 tx=0x7f9468008030 comp rx=0 tx=0).stop 2023-06-23T18:36:38.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.609+0000 7f9479650700 1 -- 172.21.15.79:0/648427217 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f947415c420 msgr2=0x7f94740b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.609+0000 7f9479650700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f947415c420 0x7f94740b3740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f946c00cd70 tx=0x7f946c00a570 comp rx=0 tx=0).stop 2023-06-23T18:36:38.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.609+0000 7f9479650700 1 -- 172.21.15.79:0/648427217 shutdown_connections 2023-06-23T18:36:38.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.609+0000 7f9479650700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f947415c420 0x7f94740b3740 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.609+0000 7f9479650700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f946006ace0 0x7f946006d190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.609+0000 7f9479650700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94740caf40 0x7f947414dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.609+0000 7f9479650700 1 --2- 172.21.15.79:0/648427217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94740ce730 0x7f94740c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.609+0000 7f9479650700 1 -- 172.21.15.79:0/648427217 >> 172.21.15.79:0/648427217 conn(0x7f947406e540 msgr2=0x7f947406e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:38.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.609+0000 7f9479650700 1 -- 172.21.15.79:0/648427217 shutdown_connections 2023-06-23T18:36:38.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.609+0000 7f9479650700 1 -- 172.21.15.79:0/648427217 wait complete. 2023-06-23T18:36:38.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:38.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.744+0000 7f43637ca700 1 Processor -- start 2023-06-23T18:36:38.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.744+0000 7f43637ca700 1 -- start start 2023-06-23T18:36:38.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f43637ca700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f435c1569c0 0x7f435c1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f43637ca700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f435c0cd440 0x7f435c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f43637ca700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f435c15d580 0x7f435c06a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f43637ca700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435c0cf570 con 0x7f435c0cd440 2023-06-23T18:36:38.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f43637ca700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435c15c8b0 con 0x7f435c1569c0 2023-06-23T18:36:38.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f43637ca700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435c160df0 con 0x7f435c15d580 2023-06-23T18:36:38.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f4360d65700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f435c0cd440 0x7f435c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f4360d65700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f435c0cd440 0x7f435c069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40788/0 (socket says 172.21.15.79:40788) 2023-06-23T18:36:38.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f4360d65700 1 -- 172.21.15.79:0/740800396 learned_addr learned my addr 172.21.15.79:0/740800396 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:38.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f4360d65700 1 -- 172.21.15.79:0/740800396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f435c15d580 msgr2=0x7f435c06a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:38.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f4360d65700 1 --2- 172.21.15.79:0/740800396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f435c15d580 0x7f435c06a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f4360d65700 1 -- 172.21.15.79:0/740800396 >> v2:172.21.15.155:3300/0 conn(0x7f435c1569c0 msgr2=0x7f435c1608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:38.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f4361566700 1 --2- 172.21.15.79:0/740800396 >> v2:172.21.15.155:3300/0 conn(0x7f435c1569c0 0x7f435c1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f4360d65700 1 --2- 172.21.15.79:0/740800396 >> v2:172.21.15.155:3300/0 conn(0x7f435c1569c0 0x7f435c1608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f4360d65700 1 -- 172.21.15.79:0/740800396 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c0cdc90 con 0x7f435c0cd440 2023-06-23T18:36:38.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.745+0000 7f4360d65700 1 --2- 172.21.15.79:0/740800396 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0cd440 0x7f435c069e90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f435c070f20 tx=0x7f434c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7119f509f9e6af26 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:38.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.746+0000 7f4353fff700 1 -- 172.21.15.79:0/740800396 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c00b870 con 0x7f435c0cd440 2023-06-23T18:36:38.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.746+0000 7f4353fff700 1 -- 172.21.15.79:0/740800396 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c00b9d0 con 0x7f435c0cd440 2023-06-23T18:36:38.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.746+0000 7f4353fff700 1 -- 172.21.15.79:0/740800396 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c00bd60 con 0x7f435c0cd440 2023-06-23T18:36:38.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.746+0000 7f43637ca700 1 -- 172.21.15.79:0/740800396 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0cd440 msgr2=0x7f435c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.746+0000 7f43637ca700 1 --2- 172.21.15.79:0/740800396 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0cd440 0x7f435c069e90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f435c070f20 tx=0x7f434c009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.746+0000 7f43637ca700 1 -- 172.21.15.79:0/740800396 shutdown_connections 2023-06-23T18:36:38.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.746+0000 7f43637ca700 1 --2- 172.21.15.79:0/740800396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f435c15d580 0x7f435c06a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.746+0000 7f43637ca700 1 --2- 172.21.15.79:0/740800396 >> v2:172.21.15.155:3300/0 conn(0x7f435c1569c0 0x7f435c1608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.746+0000 7f43637ca700 1 --2- 172.21.15.79:0/740800396 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0cd440 0x7f435c069e90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.746+0000 7f43637ca700 1 -- 172.21.15.79:0/740800396 >> 172.21.15.79:0/740800396 conn(0x7f435c06afa0 msgr2=0x7f435c06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:38.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.746+0000 7f43637ca700 1 -- 172.21.15.79:0/740800396 shutdown_connections 2023-06-23T18:36:38.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.746+0000 7f43637ca700 1 -- 172.21.15.79:0/740800396 wait complete. 2023-06-23T18:36:38.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.747+0000 7f43637ca700 1 Processor -- start 2023-06-23T18:36:38.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.747+0000 7f43637ca700 1 -- start start 2023-06-23T18:36:38.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.748+0000 7f43637ca700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f435c0cd440 0x7f435c0b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.748+0000 7f43637ca700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f435c1569c0 0x7f435c0abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.748+0000 7f43637ca700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c15d580 0x7f435c0ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.748+0000 7f43637ca700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c155610 con 0x7f435c15d580 2023-06-23T18:36:38.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.748+0000 7f43637ca700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c0c33a0 con 0x7f435c1569c0 2023-06-23T18:36:38.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.748+0000 7f43637ca700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435c0c35f0 con 0x7f435c0cd440 2023-06-23T18:36:38.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.748+0000 7f4361d67700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c15d580 0x7f435c0ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.748+0000 7f4361d67700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c15d580 0x7f435c0ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40798/0 (socket says 172.21.15.79:40798) 2023-06-23T18:36:38.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.748+0000 7f4361d67700 1 -- 172.21.15.79:0/2130345983 learned_addr learned my addr 172.21.15.79:0/2130345983 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:38.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.748+0000 7f4361566700 1 --2- 172.21.15.79:0/2130345983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f435c0cd440 0x7f435c0b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.748+0000 7f4360d65700 1 --2- 172.21.15.79:0/2130345983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f435c1569c0 0x7f435c0abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.749+0000 7f4361d67700 1 -- 172.21.15.79:0/2130345983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f435c0cd440 msgr2=0x7f435c0b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.749+0000 7f4361d67700 1 --2- 172.21.15.79:0/2130345983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f435c0cd440 0x7f435c0b5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.749+0000 7f4361d67700 1 -- 172.21.15.79:0/2130345983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f435c1569c0 msgr2=0x7f435c0abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.749+0000 7f4361d67700 1 --2- 172.21.15.79:0/2130345983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f435c1569c0 0x7f435c0abd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.749+0000 7f4361d67700 1 -- 172.21.15.79:0/2130345983 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c009550 con 0x7f435c15d580 2023-06-23T18:36:38.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.749+0000 7f4361d67700 1 --2- 172.21.15.79:0/2130345983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c15d580 0x7f435c0ac2d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f435400b970 tx=0x7f435400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:38.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.749+0000 7f4351ffb700 1 -- 172.21.15.79:0/2130345983 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435400d860 con 0x7f435c15d580 2023-06-23T18:36:38.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.749+0000 7f4351ffb700 1 -- 172.21.15.79:0/2130345983 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435400d9e0 con 0x7f435c15d580 2023-06-23T18:36:38.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.749+0000 7f43637ca700 1 -- 172.21.15.79:0/2130345983 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435c0c38d0 con 0x7f435c15d580 2023-06-23T18:36:38.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.749+0000 7f4351ffb700 1 -- 172.21.15.79:0/2130345983 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435400dd90 con 0x7f435c15d580 2023-06-23T18:36:38.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.749+0000 7f43637ca700 1 -- 172.21.15.79:0/2130345983 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435c155f60 con 0x7f435c15d580 2023-06-23T18:36:38.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.750+0000 7f43637ca700 1 -- 172.21.15.79:0/2130345983 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435c0af5d0 con 0x7f435c15d580 2023-06-23T18:36:38.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.750+0000 7f4351ffb700 1 -- 172.21.15.79:0/2130345983 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f435400e430 con 0x7f435c15d580 2023-06-23T18:36:38.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.751+0000 7f4351ffb700 1 --2- 172.21.15.79:0/2130345983 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f434006ace0 0x7f434006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:38.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.751+0000 7f4351ffb700 1 -- 172.21.15.79:0/2130345983 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4354089590 con 0x7f435c15d580 2023-06-23T18:36:38.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.751+0000 7f4361566700 1 --2- 172.21.15.79:0/2130345983 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f434006ace0 0x7f434006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:38.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.751+0000 7f4361566700 1 --2- 172.21.15.79:0/2130345983 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f434006ace0 0x7f434006d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f435800ab60 tx=0x7f4358008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:38.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.755+0000 7f4351ffb700 1 -- 172.21.15.79:0/2130345983 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f435c0af5d0 con 0x7f435c15d580 2023-06-23T18:36:38.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.940+0000 7f43637ca700 1 -- 172.21.15.79:0/2130345983 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f435c0af5d0 con 0x7f435c15d580 2023-06-23T18:36:38.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.940+0000 7f4351ffb700 1 -- 172.21.15.79:0/2130345983 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f435c0af5d0 con 0x7f435c15d580 2023-06-23T18:36:38.946 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:38.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.943+0000 7f43637ca700 1 -- 172.21.15.79:0/2130345983 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f434006ace0 msgr2=0x7f434006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.943+0000 7f43637ca700 1 --2- 172.21.15.79:0/2130345983 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f434006ace0 0x7f434006d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f435800ab60 tx=0x7f4358008040 comp rx=0 tx=0).stop 2023-06-23T18:36:38.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.943+0000 7f43637ca700 1 -- 172.21.15.79:0/2130345983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c15d580 msgr2=0x7f435c0ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:38.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.943+0000 7f43637ca700 1 --2- 172.21.15.79:0/2130345983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c15d580 0x7f435c0ac2d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f435400b970 tx=0x7f435400bc80 comp rx=0 tx=0).stop 2023-06-23T18:36:38.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.943+0000 7f43637ca700 1 -- 172.21.15.79:0/2130345983 shutdown_connections 2023-06-23T18:36:38.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.943+0000 7f43637ca700 1 --2- 172.21.15.79:0/2130345983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c15d580 0x7f435c0ac2d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.943+0000 7f43637ca700 1 --2- 172.21.15.79:0/2130345983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f435c0cd440 0x7f435c0b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.943+0000 7f43637ca700 1 --2- 172.21.15.79:0/2130345983 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f434006ace0 0x7f434006d190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.943+0000 7f43637ca700 1 --2- 172.21.15.79:0/2130345983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f435c1569c0 0x7f435c0abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:38.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.943+0000 7f43637ca700 1 -- 172.21.15.79:0/2130345983 >> 172.21.15.79:0/2130345983 conn(0x7f435c06afa0 msgr2=0x7f435c15ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:38.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.943+0000 7f43637ca700 1 -- 172.21.15.79:0/2130345983 shutdown_connections 2023-06-23T18:36:38.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:38.943+0000 7f43637ca700 1 -- 172.21.15.79:0/2130345983 wait complete. 2023-06-23T18:36:38.960 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:36:39.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.075+0000 7f64bcb3f700 1 Processor -- start 2023-06-23T18:36:39.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.075+0000 7f64bcb3f700 1 -- start start 2023-06-23T18:36:39.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.076+0000 7f64bcb3f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f64b8159170 0x7f64b80cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.076+0000 7f64bcb3f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64b80cd900 0x7f64b80ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.076+0000 7f64bcb3f700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b815cd50 con 0x7f64b80ce620 2023-06-23T18:36:39.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.076+0000 7f64bcb3f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b80cd1e0 con 0x7f64b8159170 2023-06-23T18:36:39.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.076+0000 7f64bcb3f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80cdda0 con 0x7f64b80cd900 2023-06-23T18:36:39.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.076+0000 7f64b659c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f64b8159170 0x7f64b80cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.077+0000 7f64b659c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f64b8159170 0x7f64b80cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40802/0 (socket says 172.21.15.79:40802) 2023-06-23T18:36:39.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.077+0000 7f64b659c700 1 -- 172.21.15.79:0/431107940 learned_addr learned my addr 172.21.15.79:0/431107940 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:39.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.077+0000 7f64b5d9b700 1 --2- 172.21.15.79:0/431107940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64b80cd900 0x7f64b80ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.077+0000 7f64b659c700 1 -- 172.21.15.79:0/431107940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64b80cd900 msgr2=0x7f64b80ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:39.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.077+0000 7f64b659c700 1 --2- 172.21.15.79:0/431107940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64b80cd900 0x7f64b80ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.077+0000 7f64b659c700 1 -- 172.21.15.79:0/431107940 >> v1:172.21.15.155:6789/0 conn(0x7f64b80ce620 legacy=0x7f64b8071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:39.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.077+0000 7f64b659c700 1 -- 172.21.15.79:0/431107940 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b815cfa0 con 0x7f64b8159170 2023-06-23T18:36:39.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.077+0000 7f64b5d9b700 1 --2- 172.21.15.79:0/431107940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64b80cd900 0x7f64b80ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:39.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.077+0000 7f64b659c700 1 --2- 172.21.15.79:0/431107940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64b8159170 0x7f64b80cd3c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f64a80098f0 tx=0x7f64a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28aea1551f09ce99 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:39.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.077+0000 7f64b559a700 1 -- 172.21.15.79:0/431107940 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8004230 con 0x7f64b8159170 2023-06-23T18:36:39.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.077+0000 7f64b559a700 1 -- 172.21.15.79:0/431107940 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a8004390 con 0x7f64b8159170 2023-06-23T18:36:39.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.077+0000 7f64b559a700 1 -- 172.21.15.79:0/431107940 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a80036a0 con 0x7f64b8159170 2023-06-23T18:36:39.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.078+0000 7f64bcb3f700 1 -- 172.21.15.79:0/431107940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64b8159170 msgr2=0x7f64b80cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:39.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.078+0000 7f64bcb3f700 1 --2- 172.21.15.79:0/431107940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64b8159170 0x7f64b80cd3c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f64a80098f0 tx=0x7f64a8009c00 comp rx=0 tx=0).stop 2023-06-23T18:36:39.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.078+0000 7f64bcb3f700 1 -- 172.21.15.79:0/431107940 shutdown_connections 2023-06-23T18:36:39.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.078+0000 7f64bcb3f700 1 --2- 172.21.15.79:0/431107940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64b8159170 0x7f64b80cd3c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.078+0000 7f64bcb3f700 1 --2- 172.21.15.79:0/431107940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64b80cd900 0x7f64b80ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.078+0000 7f64bcb3f700 1 -- 172.21.15.79:0/431107940 >> 172.21.15.79:0/431107940 conn(0x7f64b806d660 msgr2=0x7f64b806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:39.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.078+0000 7f64bcb3f700 1 -- 172.21.15.79:0/431107940 shutdown_connections 2023-06-23T18:36:39.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.078+0000 7f64bcb3f700 1 -- 172.21.15.79:0/431107940 wait complete. 2023-06-23T18:36:39.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.079+0000 7f64bcb3f700 1 Processor -- start 2023-06-23T18:36:39.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.079+0000 7f64bcb3f700 1 -- start start 2023-06-23T18:36:39.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64bcb3f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64b80cd900 0x7f64b815a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64bcb3f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64b80ce620 0x7f64b81518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64bcb3f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64b8159170 0x7f64b8157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64bcb3f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b8159920 con 0x7f64b8159170 2023-06-23T18:36:39.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64bcb3f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b8159c00 con 0x7f64b80ce620 2023-06-23T18:36:39.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64bcb3f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b8159ee0 con 0x7f64b80cd900 2023-06-23T18:36:39.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64b6d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64b8159170 0x7f64b8157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64b6d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64b8159170 0x7f64b8157690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40812/0 (socket says 172.21.15.79:40812) 2023-06-23T18:36:39.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64b6d9d700 1 -- 172.21.15.79:0/1476360582 learned_addr learned my addr 172.21.15.79:0/1476360582 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:39.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64b5d9b700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64b80ce620 0x7f64b81518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64b659c700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64b80cd900 0x7f64b815a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64b6d9d700 1 -- 172.21.15.79:0/1476360582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64b80cd900 msgr2=0x7f64b815a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:39.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64b6d9d700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64b80cd900 0x7f64b815a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64b6d9d700 1 -- 172.21.15.79:0/1476360582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64b80ce620 msgr2=0x7f64b81518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:39.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64b6d9d700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64b80ce620 0x7f64b81518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64b6d9d700 1 -- 172.21.15.79:0/1476360582 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a8009550 con 0x7f64b8159170 2023-06-23T18:36:39.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64b659c700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64b80cd900 0x7f64b815a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:39.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64b5d9b700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64b80ce620 0x7f64b81518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:39.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64b6d9d700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64b8159170 0x7f64b8157690 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f64b000cc10 tx=0x7f64b000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:39.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.080+0000 7f64aeffd700 1 -- 172.21.15.79:0/1476360582 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b000abb0 con 0x7f64b8159170 2023-06-23T18:36:39.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.081+0000 7f64aeffd700 1 -- 172.21.15.79:0/1476360582 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b000ad10 con 0x7f64b8159170 2023-06-23T18:36:39.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.081+0000 7f64bcb3f700 1 -- 172.21.15.79:0/1476360582 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b815a1c0 con 0x7f64b8159170 2023-06-23T18:36:39.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.081+0000 7f64aeffd700 1 -- 172.21.15.79:0/1476360582 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0012660 con 0x7f64b8159170 2023-06-23T18:36:39.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.081+0000 7f64bcb3f700 1 -- 172.21.15.79:0/1476360582 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b815af00 con 0x7f64b8159170 2023-06-23T18:36:39.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.081+0000 7f64bcb3f700 1 -- 172.21.15.79:0/1476360582 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b815a6e0 con 0x7f64b8159170 2023-06-23T18:36:39.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.082+0000 7f64aeffd700 1 -- 172.21.15.79:0/1476360582 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f64b00127c0 con 0x7f64b8159170 2023-06-23T18:36:39.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.082+0000 7f64aeffd700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f649806ace0 0x7f649806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.082+0000 7f64aeffd700 1 -- 172.21.15.79:0/1476360582 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f64b0089c00 con 0x7f64b8159170 2023-06-23T18:36:39.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.082+0000 7f64b659c700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f649806ace0 0x7f649806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.083+0000 7f64b659c700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f649806ace0 0x7f649806d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009c40 tx=0x7f64a8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:39.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.087+0000 7f64aeffd700 1 -- 172.21.15.79:0/1476360582 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f64b815a6e0 con 0x7f64b8159170 2023-06-23T18:36:39.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.290+0000 7f64bcb3f700 1 -- 172.21.15.79:0/1476360582 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f64b815a6e0 con 0x7f64b8159170 2023-06-23T18:36:39.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.291+0000 7f64aeffd700 1 -- 172.21.15.79:0/1476360582 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f64b815a6e0 con 0x7f64b8159170 2023-06-23T18:36:39.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.293+0000 7f64bcb3f700 1 -- 172.21.15.79:0/1476360582 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f649806ace0 msgr2=0x7f649806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:39.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.293+0000 7f64bcb3f700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f649806ace0 0x7f649806d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009c40 tx=0x7f64a8008030 comp rx=0 tx=0).stop 2023-06-23T18:36:39.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.293+0000 7f64bcb3f700 1 -- 172.21.15.79:0/1476360582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64b8159170 msgr2=0x7f64b8157690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:39.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.293+0000 7f64bcb3f700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64b8159170 0x7f64b8157690 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f64b000cc10 tx=0x7f64b000a060 comp rx=0 tx=0).stop 2023-06-23T18:36:39.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.293+0000 7f64bcb3f700 1 -- 172.21.15.79:0/1476360582 shutdown_connections 2023-06-23T18:36:39.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.293+0000 7f64bcb3f700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64b8159170 0x7f64b8157690 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.293+0000 7f64bcb3f700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64b80cd900 0x7f64b815a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.293+0000 7f64bcb3f700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f649806ace0 0x7f649806d190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.293+0000 7f64bcb3f700 1 --2- 172.21.15.79:0/1476360582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64b80ce620 0x7f64b81518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.293+0000 7f64bcb3f700 1 -- 172.21.15.79:0/1476360582 >> 172.21.15.79:0/1476360582 conn(0x7f64b806d660 msgr2=0x7f64b8069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:39.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.293+0000 7f64bcb3f700 1 -- 172.21.15.79:0/1476360582 shutdown_connections 2023-06-23T18:36:39.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.293+0000 7f64bcb3f700 1 -- 172.21.15.79:0/1476360582 wait complete. 2023-06-23T18:36:39.300 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-23T18:36:39.310 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi079.front.sepia.ceph.com... 2023-06-23T18:36:39.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:39.312 DEBUG:teuthology.orchestra.run.smithi079:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-06-23T18:36:39.346 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:36:39.346 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:36:39.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:39.347 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-06-23T18:36:39.403 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:36:39.403 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:36:39.425 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:36:39.426 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:36:39.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:39.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.600+0000 7f1963c23700 1 Processor -- start 2023-06-23T18:36:39.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.600+0000 7f1963c23700 1 -- start start 2023-06-23T18:36:39.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f1963c23700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f195c159170 0x7f195c0cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f1963c23700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f195c0cd900 0x7f195c0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f1963c23700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f195c0ce620 0x7f195c071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f1963c23700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195c0cdd80 con 0x7f195c159170 2023-06-23T18:36:39.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f1963c23700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195c15cd50 con 0x7f195c0cd900 2023-06-23T18:36:39.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f1963c23700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c15cfa0 con 0x7f195c0ce620 2023-06-23T18:36:39.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f19619bf700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f195c159170 0x7f195c0cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f19619bf700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f195c159170 0x7f195c0cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40816/0 (socket says 172.21.15.79:40816) 2023-06-23T18:36:39.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f19619bf700 1 -- 172.21.15.79:0/967685968 learned_addr learned my addr 172.21.15.79:0/967685968 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:39.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f19611be700 1 --2- 172.21.15.79:0/967685968 >> v2:172.21.15.155:3300/0 conn(0x7f195c0cd900 0x7f195c0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f19619bf700 1 -- 172.21.15.79:0/967685968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f195c0ce620 msgr2=0x7f195c071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:39.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f19621c0700 1 --2- 172.21.15.79:0/967685968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f195c0ce620 0x7f195c071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f19619bf700 1 --2- 172.21.15.79:0/967685968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f195c0ce620 0x7f195c071490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f19619bf700 1 -- 172.21.15.79:0/967685968 >> v2:172.21.15.155:3300/0 conn(0x7f195c0cd900 msgr2=0x7f195c0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:39.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f19619bf700 1 --2- 172.21.15.79:0/967685968 >> v2:172.21.15.155:3300/0 conn(0x7f195c0cd900 0x7f195c0ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f19619bf700 1 -- 172.21.15.79:0/967685968 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c15d280 con 0x7f195c159170 2023-06-23T18:36:39.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f19621c0700 1 --2- 172.21.15.79:0/967685968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f195c0ce620 0x7f195c071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:39.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f19619bf700 1 --2- 172.21.15.79:0/967685968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f195c159170 0x7f195c0cd3c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f195c070f20 tx=0x7f1958009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41510a0ed9939d49 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:39.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.601+0000 7f19609bd700 1 -- 172.21.15.79:0/967685968 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195800b870 con 0x7f195c159170 2023-06-23T18:36:39.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.602+0000 7f19609bd700 1 -- 172.21.15.79:0/967685968 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195800b9d0 con 0x7f195c159170 2023-06-23T18:36:39.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.602+0000 7f19609bd700 1 -- 172.21.15.79:0/967685968 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195800bd60 con 0x7f195c159170 2023-06-23T18:36:39.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.602+0000 7f1963c23700 1 -- 172.21.15.79:0/967685968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f195c159170 msgr2=0x7f195c0cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:39.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.602+0000 7f1963c23700 1 --2- 172.21.15.79:0/967685968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f195c159170 0x7f195c0cd3c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f195c070f20 tx=0x7f1958009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.602+0000 7f1963c23700 1 -- 172.21.15.79:0/967685968 shutdown_connections 2023-06-23T18:36:39.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.602+0000 7f1963c23700 1 --2- 172.21.15.79:0/967685968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f195c0ce620 0x7f195c071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.602+0000 7f1963c23700 1 --2- 172.21.15.79:0/967685968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f195c159170 0x7f195c0cd3c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.602+0000 7f1963c23700 1 --2- 172.21.15.79:0/967685968 >> v2:172.21.15.155:3300/0 conn(0x7f195c0cd900 0x7f195c0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.602+0000 7f1963c23700 1 -- 172.21.15.79:0/967685968 >> 172.21.15.79:0/967685968 conn(0x7f195c06d660 msgr2=0x7f195c06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:39.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.602+0000 7f1963c23700 1 -- 172.21.15.79:0/967685968 shutdown_connections 2023-06-23T18:36:39.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.602+0000 7f1963c23700 1 -- 172.21.15.79:0/967685968 wait complete. 2023-06-23T18:36:39.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.604+0000 7f1963c23700 1 Processor -- start 2023-06-23T18:36:39.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.604+0000 7f1963c23700 1 -- start start 2023-06-23T18:36:39.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f1963c23700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f195c0cd900 0x7f195c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f1963c23700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f195c0ce620 0x7f195c14e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f1963c23700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f195c159170 0x7f195c159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f1963c23700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c15c700 con 0x7f195c0cd900 2023-06-23T18:36:39.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f1963c23700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c14e5d0 con 0x7f195c0ce620 2023-06-23T18:36:39.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f1963c23700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c0b5170 con 0x7f195c159170 2023-06-23T18:36:39.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f19619bf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f195c0cd900 0x7f195c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f19619bf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f195c0cd900 0x7f195c15c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40822/0 (socket says 172.21.15.79:40822) 2023-06-23T18:36:39.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f19619bf700 1 -- 172.21.15.79:0/644616148 learned_addr learned my addr 172.21.15.79:0/644616148 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:39.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f19619bf700 1 -- 172.21.15.79:0/644616148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f195c159170 msgr2=0x7f195c159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:39.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f19619bf700 1 --2- 172.21.15.79:0/644616148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f195c159170 0x7f195c159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f19619bf700 1 -- 172.21.15.79:0/644616148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f195c0ce620 msgr2=0x7f195c14e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:39.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f19619bf700 1 --2- 172.21.15.79:0/644616148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f195c0ce620 0x7f195c14e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f19619bf700 1 -- 172.21.15.79:0/644616148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1958009550 con 0x7f195c0cd900 2023-06-23T18:36:39.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f19619bf700 1 --2- 172.21.15.79:0/644616148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f195c0cd900 0x7f195c15c1c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1958000c00 tx=0x7f1958004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:39.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f19527fc700 1 -- 172.21.15.79:0/644616148 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958004520 con 0x7f195c0cd900 2023-06-23T18:36:39.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f19527fc700 1 -- 172.21.15.79:0/644616148 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195801e070 con 0x7f195c0cd900 2023-06-23T18:36:39.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f19527fc700 1 -- 172.21.15.79:0/644616148 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19580038c0 con 0x7f195c0cd900 2023-06-23T18:36:39.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f1963c23700 1 -- 172.21.15.79:0/644616148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195c0c4540 con 0x7f195c0cd900 2023-06-23T18:36:39.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.605+0000 7f1963c23700 1 -- 172.21.15.79:0/644616148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195c0c4b00 con 0x7f195c0cd900 2023-06-23T18:36:39.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.606+0000 7f1963c23700 1 -- 172.21.15.79:0/644616148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195c04e240 con 0x7f195c0cd900 2023-06-23T18:36:39.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.607+0000 7f19527fc700 1 -- 172.21.15.79:0/644616148 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1958003a20 con 0x7f195c0cd900 2023-06-23T18:36:39.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.607+0000 7f19527fc700 1 --2- 172.21.15.79:0/644616148 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f194006ad10 0x7f194006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.608+0000 7f19527fc700 1 -- 172.21.15.79:0/644616148 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1958020070 con 0x7f195c0cd900 2023-06-23T18:36:39.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.608+0000 7f19611be700 1 --2- 172.21.15.79:0/644616148 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f194006ad10 0x7f194006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.608+0000 7f19611be700 1 --2- 172.21.15.79:0/644616148 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f194006ad10 0x7f194006d1c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f195c071b70 tx=0x7f194c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:39.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.611+0000 7f19527fc700 1 -- 172.21.15.79:0/644616148 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1958055cd0 con 0x7f195c0cd900 2023-06-23T18:36:39.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.799+0000 7f1963c23700 1 -- 172.21.15.79:0/644616148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f195c0c3030 con 0x7f195c0cd900 2023-06-23T18:36:39.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.800+0000 7f19527fc700 1 -- 172.21.15.79:0/644616148 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18233 (secure 0 0 0) 0x7f19580592f0 con 0x7f195c0cd900 2023-06-23T18:36:39.805 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:39.805 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:35:54.513520+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:39.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.802+0000 7f1963c23700 1 -- 172.21.15.79:0/644616148 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f194006ad10 msgr2=0x7f194006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:39.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.802+0000 7f1963c23700 1 --2- 172.21.15.79:0/644616148 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f194006ad10 0x7f194006d1c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f195c071b70 tx=0x7f194c008040 comp rx=0 tx=0).stop 2023-06-23T18:36:39.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.802+0000 7f1963c23700 1 -- 172.21.15.79:0/644616148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f195c0cd900 msgr2=0x7f195c15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:39.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.802+0000 7f1963c23700 1 --2- 172.21.15.79:0/644616148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f195c0cd900 0x7f195c15c1c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1958000c00 tx=0x7f1958004750 comp rx=0 tx=0).stop 2023-06-23T18:36:39.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.802+0000 7f1963c23700 1 -- 172.21.15.79:0/644616148 shutdown_connections 2023-06-23T18:36:39.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.802+0000 7f1963c23700 1 --2- 172.21.15.79:0/644616148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f195c159170 0x7f195c159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.802+0000 7f1963c23700 1 --2- 172.21.15.79:0/644616148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f195c0cd900 0x7f195c15c1c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.802+0000 7f1963c23700 1 --2- 172.21.15.79:0/644616148 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f194006ad10 0x7f194006d1c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.802+0000 7f1963c23700 1 --2- 172.21.15.79:0/644616148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f195c0ce620 0x7f195c14e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.802+0000 7f1963c23700 1 -- 172.21.15.79:0/644616148 >> 172.21.15.79:0/644616148 conn(0x7f195c06d660 msgr2=0x7f195c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:39.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.802+0000 7f1963c23700 1 -- 172.21.15.79:0/644616148 shutdown_connections 2023-06-23T18:36:39.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.802+0000 7f1963c23700 1 -- 172.21.15.79:0/644616148 wait complete. 2023-06-23T18:36:39.820 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:39.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:39.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.938+0000 7fd60b749700 1 Processor -- start 2023-06-23T18:36:39.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.939+0000 7fd60b749700 1 -- start start 2023-06-23T18:36:39.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.939+0000 7fd60b749700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd6041569c0 0x7fd6041608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.939+0000 7fd60b749700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd6040cd440 0x7fd604069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.939+0000 7fd60b749700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd60415d580 0x7fd60406a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd60b749700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6040cf570 con 0x7fd6041569c0 2023-06-23T18:36:39.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd60b749700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd60415c8b0 con 0x7fd6040cd440 2023-06-23T18:36:39.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd60b749700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd604160df0 con 0x7fd60415d580 2023-06-23T18:36:39.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd6094e5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd6041569c0 0x7fd6041608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd6094e5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd6041569c0 0x7fd6041608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40830/0 (socket says 172.21.15.79:40830) 2023-06-23T18:36:39.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd6094e5700 1 -- 172.21.15.79:0/3174221400 learned_addr learned my addr 172.21.15.79:0/3174221400 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:39.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd609ce6700 1 --2- 172.21.15.79:0/3174221400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd60415d580 0x7fd60406a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd608ce4700 1 --2- 172.21.15.79:0/3174221400 >> v2:172.21.15.155:3300/0 conn(0x7fd6040cd440 0x7fd604069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd6094e5700 1 -- 172.21.15.79:0/3174221400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd60415d580 msgr2=0x7fd60406a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:39.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd6094e5700 1 --2- 172.21.15.79:0/3174221400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd60415d580 0x7fd60406a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd6094e5700 1 -- 172.21.15.79:0/3174221400 >> v2:172.21.15.155:3300/0 conn(0x7fd6040cd440 msgr2=0x7fd604069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:39.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd6094e5700 1 --2- 172.21.15.79:0/3174221400 >> v2:172.21.15.155:3300/0 conn(0x7fd6040cd440 0x7fd604069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd6094e5700 1 -- 172.21.15.79:0/3174221400 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6040cdc90 con 0x7fd6041569c0 2023-06-23T18:36:39.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd6094e5700 1 --2- 172.21.15.79:0/3174221400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6041569c0 0x7fd6041608b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd6040702d0 tx=0x7fd5f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd15c7ad8d7d3591 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:39.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd5fbfff700 1 -- 172.21.15.79:0/3174221400 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400b870 con 0x7fd6041569c0 2023-06-23T18:36:39.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd5fbfff700 1 -- 172.21.15.79:0/3174221400 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f400b9d0 con 0x7fd6041569c0 2023-06-23T18:36:39.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.940+0000 7fd5fbfff700 1 -- 172.21.15.79:0/3174221400 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400bd60 con 0x7fd6041569c0 2023-06-23T18:36:39.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.941+0000 7fd60b749700 1 -- 172.21.15.79:0/3174221400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6041569c0 msgr2=0x7fd6041608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:39.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.941+0000 7fd60b749700 1 --2- 172.21.15.79:0/3174221400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6041569c0 0x7fd6041608b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd6040702d0 tx=0x7fd5f4009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.941+0000 7fd60b749700 1 -- 172.21.15.79:0/3174221400 shutdown_connections 2023-06-23T18:36:39.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.941+0000 7fd60b749700 1 --2- 172.21.15.79:0/3174221400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd60415d580 0x7fd60406a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.953 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.941+0000 7fd60b749700 1 --2- 172.21.15.79:0/3174221400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6041569c0 0x7fd6041608b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.953 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.941+0000 7fd60b749700 1 --2- 172.21.15.79:0/3174221400 >> v2:172.21.15.155:3300/0 conn(0x7fd6040cd440 0x7fd604069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.953 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.941+0000 7fd60b749700 1 -- 172.21.15.79:0/3174221400 >> 172.21.15.79:0/3174221400 conn(0x7fd60406afa0 msgr2=0x7fd60406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:39.953 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.941+0000 7fd60b749700 1 -- 172.21.15.79:0/3174221400 shutdown_connections 2023-06-23T18:36:39.953 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.941+0000 7fd60b749700 1 -- 172.21.15.79:0/3174221400 wait complete. 2023-06-23T18:36:39.955 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.942+0000 7fd60b749700 1 Processor -- start 2023-06-23T18:36:39.955 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.942+0000 7fd60b749700 1 -- start start 2023-06-23T18:36:39.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd60b749700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6040cd440 0x7fd6040b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd60b749700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6041569c0 0x7fd6041f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd60b749700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd60415d580 0x7fd6041f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd60b749700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6040b3000 con 0x7fd6040cd440 2023-06-23T18:36:39.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd60b749700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6040b13a0 con 0x7fd6041569c0 2023-06-23T18:36:39.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd60b749700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6040b1680 con 0x7fd60415d580 2023-06-23T18:36:39.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd6094e5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6040cd440 0x7fd6040b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd6094e5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6040cd440 0x7fd6040b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40836/0 (socket says 172.21.15.79:40836) 2023-06-23T18:36:39.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd6094e5700 1 -- 172.21.15.79:0/2382238459 learned_addr learned my addr 172.21.15.79:0/2382238459 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:39.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd6094e5700 1 -- 172.21.15.79:0/2382238459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd60415d580 msgr2=0x7fd6041f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:39.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd6094e5700 1 --2- 172.21.15.79:0/2382238459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd60415d580 0x7fd6041f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd6094e5700 1 -- 172.21.15.79:0/2382238459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6041569c0 msgr2=0x7fd6041f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:39.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd6094e5700 1 --2- 172.21.15.79:0/2382238459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6041569c0 0x7fd6041f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:39.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd6094e5700 1 -- 172.21.15.79:0/2382238459 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f4009550 con 0x7fd6040cd440 2023-06-23T18:36:39.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.943+0000 7fd6094e5700 1 --2- 172.21.15.79:0/2382238459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6040cd440 0x7fd6040b0950 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4000c00 tx=0x7fd5f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:39.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.944+0000 7fd5f9ffb700 1 -- 172.21.15.79:0/2382238459 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4004540 con 0x7fd6040cd440 2023-06-23T18:36:39.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.944+0000 7fd5f9ffb700 1 -- 172.21.15.79:0/2382238459 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f40036a0 con 0x7fd6040cd440 2023-06-23T18:36:39.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.944+0000 7fd5f9ffb700 1 -- 172.21.15.79:0/2382238459 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4003a50 con 0x7fd6040cd440 2023-06-23T18:36:39.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.944+0000 7fd60b749700 1 -- 172.21.15.79:0/2382238459 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6040c38f0 con 0x7fd6040cd440 2023-06-23T18:36:39.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.944+0000 7fd60b749700 1 -- 172.21.15.79:0/2382238459 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6040b2880 con 0x7fd6040cd440 2023-06-23T18:36:39.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.945+0000 7fd60b749700 1 -- 172.21.15.79:0/2382238459 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6040c3e10 con 0x7fd6040cd440 2023-06-23T18:36:39.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.945+0000 7fd5f9ffb700 1 -- 172.21.15.79:0/2382238459 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd5f4021070 con 0x7fd6040cd440 2023-06-23T18:36:39.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.945+0000 7fd5f9ffb700 1 --2- 172.21.15.79:0/2382238459 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd5e806ace0 0x7fd5e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:39.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.945+0000 7fd5f9ffb700 1 -- 172.21.15.79:0/2382238459 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd5f408a9b0 con 0x7fd6040cd440 2023-06-23T18:36:39.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.946+0000 7fd608ce4700 1 --2- 172.21.15.79:0/2382238459 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd5e806ace0 0x7fd5e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:39.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.946+0000 7fd608ce4700 1 --2- 172.21.15.79:0/2382238459 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd5e806ace0 0x7fd5e806d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd604148960 tx=0x7fd600008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:39.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:39.950+0000 7fd5f9ffb700 1 -- 172.21.15.79:0/2382238459 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd6040c3e10 con 0x7fd6040cd440 2023-06-23T18:36:40.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.136+0000 7fd60b749700 1 -- 172.21.15.79:0/2382238459 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd6040cdc90 con 0x7fd6040cd440 2023-06-23T18:36:40.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.136+0000 7fd5f9ffb700 1 -- 172.21.15.79:0/2382238459 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd6040cdc90 con 0x7fd6040cd440 2023-06-23T18:36:40.142 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:40.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.138+0000 7fd60b749700 1 -- 172.21.15.79:0/2382238459 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd5e806ace0 msgr2=0x7fd5e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.138+0000 7fd60b749700 1 --2- 172.21.15.79:0/2382238459 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd5e806ace0 0x7fd5e806d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd604148960 tx=0x7fd600008040 comp rx=0 tx=0).stop 2023-06-23T18:36:40.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.138+0000 7fd60b749700 1 -- 172.21.15.79:0/2382238459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6040cd440 msgr2=0x7fd6040b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.138+0000 7fd60b749700 1 --2- 172.21.15.79:0/2382238459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6040cd440 0x7fd6040b0950 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4000c00 tx=0x7fd5f4004750 comp rx=0 tx=0).stop 2023-06-23T18:36:40.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.138+0000 7fd60b749700 1 -- 172.21.15.79:0/2382238459 shutdown_connections 2023-06-23T18:36:40.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.138+0000 7fd60b749700 1 --2- 172.21.15.79:0/2382238459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd60415d580 0x7fd6041f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.138+0000 7fd60b749700 1 --2- 172.21.15.79:0/2382238459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6040cd440 0x7fd6040b0950 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.138+0000 7fd60b749700 1 --2- 172.21.15.79:0/2382238459 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd5e806ace0 0x7fd5e806d190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.138+0000 7fd60b749700 1 --2- 172.21.15.79:0/2382238459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6041569c0 0x7fd6041f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.138+0000 7fd60b749700 1 -- 172.21.15.79:0/2382238459 >> 172.21.15.79:0/2382238459 conn(0x7fd60406afa0 msgr2=0x7fd6040cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:40.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.138+0000 7fd60b749700 1 -- 172.21.15.79:0/2382238459 shutdown_connections 2023-06-23T18:36:40.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.138+0000 7fd60b749700 1 -- 172.21.15.79:0/2382238459 wait complete. 2023-06-23T18:36:40.156 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:40.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.274+0000 7f79c6d53700 1 Processor -- start 2023-06-23T18:36:40.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.274+0000 7f79c6d53700 1 -- start start 2023-06-23T18:36:40.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.275+0000 7f79c6d53700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f79c01438f0 0x7f79c00b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.275+0000 7f79c6d53700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c00b0b10 con 0x7f79c00d08c0 2023-06-23T18:36:40.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.275+0000 7f79c6d53700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c00b0df0 con 0x7f79c00cedf0 2023-06-23T18:36:40.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.275+0000 7f79c6d53700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c00b10d0 con 0x7f79c01438f0 2023-06-23T18:36:40.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.275+0000 7f79c52f0700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f79c00d08c0 0x7f79c00d0cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43890/0 (socket says 172.21.15.79:43890) 2023-06-23T18:36:40.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.275+0000 7f79c52f0700 1 -- 172.21.15.79:0/1360606281 learned_addr learned my addr 172.21.15.79:0/1360606281 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:40.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.275+0000 7f79bf7fe700 1 -- 172.21.15.79:0/1360606281 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3953347461 0 0) 0x7f79c00b0b10 con 0x7f79c00d08c0 2023-06-23T18:36:40.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.275+0000 7f79bf7fe700 1 -- 172.21.15.79:0/1360606281 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79b00035b0 con 0x7f79c00d08c0 2023-06-23T18:36:40.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.275+0000 7f79bffff700 1 --2- 172.21.15.79:0/1360606281 >> v2:172.21.15.155:3300/0 conn(0x7f79c01438f0 0x7f79c00b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 -- 172.21.15.79:0/1360606281 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1864200873 0 0) 0x7f79b00035b0 con 0x7f79c00d08c0 2023-06-23T18:36:40.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 -- 172.21.15.79:0/1360606281 >> v2:172.21.15.155:3300/0 conn(0x7f79c01438f0 msgr2=0x7f79c00b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:40.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 --2- 172.21.15.79:0/1360606281 >> v2:172.21.15.155:3300/0 conn(0x7f79c01438f0 0x7f79c00b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 -- 172.21.15.79:0/1360606281 >> v1:172.21.15.155:6789/0 conn(0x7f79c00cedf0 legacy=0x7f79c00b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.285 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 -- 172.21.15.79:0/1360606281 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c00b13b0 con 0x7f79c00d08c0 2023-06-23T18:36:40.285 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 -- 172.21.15.79:0/1360606281 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f79b80019f0 con 0x7f79c00d08c0 2023-06-23T18:36:40.285 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 -- 172.21.15.79:0/1360606281 >> v1:172.21.15.79:6789/0 conn(0x7f79c00d08c0 legacy=0x7f79c00d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.285 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 --2- 172.21.15.79:0/1360606281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79b0003a00 0x7f79b0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.285 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 --2- 172.21.15.79:0/1360606281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79b0004850 0x7f79b0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.285 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 --2- 172.21.15.79:0/1360606281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79b0005200 0x7f79b000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.285 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 -- 172.21.15.79:0/1360606281 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c00b10d0 con 0x7f79b0005200 2023-06-23T18:36:40.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 -- 172.21.15.79:0/1360606281 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b000c110 con 0x7f79b0003a00 2023-06-23T18:36:40.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 -- 172.21.15.79:0/1360606281 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b000c2d0 con 0x7f79b0004850 2023-06-23T18:36:40.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79c52f0700 1 --2- 172.21.15.79:0/1360606281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79b0005200 0x7f79b000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79c4aef700 1 --2- 172.21.15.79:0/1360606281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79b0003a00 0x7f79b0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bffff700 1 --2- 172.21.15.79:0/1360606281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79b0004850 0x7f79b0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79c52f0700 1 -- 172.21.15.79:0/1360606281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79b0004850 msgr2=0x7f79b0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:40.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79c52f0700 1 --2- 172.21.15.79:0/1360606281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79b0004850 0x7f79b0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79c52f0700 1 -- 172.21.15.79:0/1360606281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79b0003a00 msgr2=0x7f79b0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:40.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79c52f0700 1 --2- 172.21.15.79:0/1360606281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79b0003a00 0x7f79b0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79c52f0700 1 -- 172.21.15.79:0/1360606281 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b00035b0 con 0x7f79b0005200 2023-06-23T18:36:40.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79c52f0700 1 -- 172.21.15.79:0/1360606281 reap_dead start 2023-06-23T18:36:40.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79c52f0700 1 --2- 172.21.15.79:0/1360606281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79b0005200 0x7f79b000ba10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f79b800a830 tx=0x7f79b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4d9d321be3b98a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:40.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 -- 172.21.15.79:0/1360606281 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b800a2f0 con 0x7f79b0005200 2023-06-23T18:36:40.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 -- 172.21.15.79:0/1360606281 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b8012600 con 0x7f79b0005200 2023-06-23T18:36:40.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.276+0000 7f79bf7fe700 1 -- 172.21.15.79:0/1360606281 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b80129b0 con 0x7f79b0005200 2023-06-23T18:36:40.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.277+0000 7f79c6d53700 1 -- 172.21.15.79:0/1360606281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79b0005200 msgr2=0x7f79b000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.277+0000 7f79c6d53700 1 --2- 172.21.15.79:0/1360606281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79b0005200 0x7f79b000ba10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f79b800a830 tx=0x7f79b0005880 comp rx=0 tx=0).stop 2023-06-23T18:36:40.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.277+0000 7f79c6d53700 1 -- 172.21.15.79:0/1360606281 shutdown_connections 2023-06-23T18:36:40.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.277+0000 7f79c6d53700 1 --2- 172.21.15.79:0/1360606281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79b0005200 0x7f79b000ba10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.277+0000 7f79c6d53700 1 -- 172.21.15.79:0/1360606281 >> 172.21.15.79:0/1360606281 conn(0x7f79c006a7b0 msgr2=0x7f79c0099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:40.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.277+0000 7f79c6d53700 1 -- 172.21.15.79:0/1360606281 shutdown_connections 2023-06-23T18:36:40.289 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.277+0000 7f79c6d53700 1 -- 172.21.15.79:0/1360606281 wait complete. 2023-06-23T18:36:40.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.278+0000 7f79c6d53700 1 Processor -- start 2023-06-23T18:36:40.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.278+0000 7f79c6d53700 1 -- start start 2023-06-23T18:36:40.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.279+0000 7f79c6d53700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79b0005200 0x7f79c0154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.279+0000 7f79c6d53700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79c006b960 0x7f79c006b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.279+0000 7f79c6d53700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c006f6d0 0x7f79c006fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.279+0000 7f79c6d53700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c00700b0 con 0x7f79c006b960 2023-06-23T18:36:40.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.279+0000 7f79c6d53700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c015b340 con 0x7f79b0005200 2023-06-23T18:36:40.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.279+0000 7f79c6d53700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c006c9e0 con 0x7f79c006f6d0 2023-06-23T18:36:40.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.279+0000 7f79bffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79c006b960 0x7f79c006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.279+0000 7f79bffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79c006b960 0x7f79c006b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40858/0 (socket says 172.21.15.79:40858) 2023-06-23T18:36:40.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.279+0000 7f79bffff700 1 -- 172.21.15.79:0/3821528105 learned_addr learned my addr 172.21.15.79:0/3821528105 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:40.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.279+0000 7f79c52f0700 1 --2- 172.21.15.79:0/3821528105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c006f6d0 0x7f79c006fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.279+0000 7f79c4aef700 1 --2- 172.21.15.79:0/3821528105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79b0005200 0x7f79c0154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.280+0000 7f79bffff700 1 -- 172.21.15.79:0/3821528105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c006f6d0 msgr2=0x7f79c006fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.280+0000 7f79bffff700 1 --2- 172.21.15.79:0/3821528105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c006f6d0 0x7f79c006fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.280+0000 7f79bffff700 1 -- 172.21.15.79:0/3821528105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79b0005200 msgr2=0x7f79c0154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.280+0000 7f79bffff700 1 --2- 172.21.15.79:0/3821528105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79b0005200 0x7f79c0154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.280+0000 7f79bffff700 1 -- 172.21.15.79:0/3821528105 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c006bdd0 con 0x7f79c006b960 2023-06-23T18:36:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.280+0000 7f79bffff700 1 --2- 172.21.15.79:0/3821528105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79c006b960 0x7f79c006b080 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0119d0 tx=0x7f79ac011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.280+0000 7f79bd7fa700 1 -- 172.21.15.79:0/3821528105 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac011e60 con 0x7f79c006b960 2023-06-23T18:36:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.280+0000 7f79bd7fa700 1 -- 172.21.15.79:0/3821528105 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ac00f940 con 0x7f79c006b960 2023-06-23T18:36:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.280+0000 7f79c6d53700 1 -- 172.21.15.79:0/3821528105 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c006c0b0 con 0x7f79c006b960 2023-06-23T18:36:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.280+0000 7f79bd7fa700 1 -- 172.21.15.79:0/3821528105 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac00fcd0 con 0x7f79c006b960 2023-06-23T18:36:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.280+0000 7f79c6d53700 1 -- 172.21.15.79:0/3821528105 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c006c670 con 0x7f79c006b960 2023-06-23T18:36:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.281+0000 7f79c6d53700 1 -- 172.21.15.79:0/3821528105 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c00b33c0 con 0x7f79c006b960 2023-06-23T18:36:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.281+0000 7f79bd7fa700 1 -- 172.21.15.79:0/3821528105 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f79ac014440 con 0x7f79c006b960 2023-06-23T18:36:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.282+0000 7f79bd7fa700 1 --2- 172.21.15.79:0/3821528105 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f79a406ace0 0x7f79a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.282+0000 7f79bd7fa700 1 -- 172.21.15.79:0/3821528105 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f79ac089a70 con 0x7f79c006b960 2023-06-23T18:36:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.282+0000 7f79c4aef700 1 --2- 172.21.15.79:0/3821528105 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f79a406ace0 0x7f79a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.282+0000 7f79c4aef700 1 --2- 172.21.15.79:0/3821528105 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f79a406ace0 0x7f79a406d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f79b4000c00 tx=0x7f79b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.286+0000 7f79bd7fa700 1 -- 172.21.15.79:0/3821528105 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f79ac0585b0 con 0x7f79c006b960 2023-06-23T18:36:40.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.473+0000 7f79c6d53700 1 -- 172.21.15.79:0/3821528105 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79c015b4b0 con 0x7f79c006b960 2023-06-23T18:36:40.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.473+0000 7f79bd7fa700 1 -- 172.21.15.79:0/3821528105 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79c015b4b0 con 0x7f79c006b960 2023-06-23T18:36:40.479 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:40.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.475+0000 7f79c6d53700 1 -- 172.21.15.79:0/3821528105 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f79a406ace0 msgr2=0x7f79a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.475+0000 7f79c6d53700 1 --2- 172.21.15.79:0/3821528105 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f79a406ace0 0x7f79a406d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f79b4000c00 tx=0x7f79b4009040 comp rx=0 tx=0).stop 2023-06-23T18:36:40.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.475+0000 7f79c6d53700 1 -- 172.21.15.79:0/3821528105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79c006b960 msgr2=0x7f79c006b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.475+0000 7f79c6d53700 1 --2- 172.21.15.79:0/3821528105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79c006b960 0x7f79c006b080 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0119d0 tx=0x7f79ac011ce0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.475+0000 7f79c6d53700 1 -- 172.21.15.79:0/3821528105 shutdown_connections 2023-06-23T18:36:40.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.475+0000 7f79c6d53700 1 --2- 172.21.15.79:0/3821528105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c006f6d0 0x7f79c006fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.475+0000 7f79c6d53700 1 --2- 172.21.15.79:0/3821528105 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f79a406ace0 0x7f79a406d190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.475+0000 7f79c6d53700 1 --2- 172.21.15.79:0/3821528105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79b0005200 0x7f79c0154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.475+0000 7f79c6d53700 1 --2- 172.21.15.79:0/3821528105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79c006b960 0x7f79c006b080 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.475+0000 7f79c6d53700 1 -- 172.21.15.79:0/3821528105 >> 172.21.15.79:0/3821528105 conn(0x7f79c006a7b0 msgr2=0x7f79c00b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:40.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.475+0000 7f79c6d53700 1 -- 172.21.15.79:0/3821528105 shutdown_connections 2023-06-23T18:36:40.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.475+0000 7f79c6d53700 1 -- 172.21.15.79:0/3821528105 wait complete. 2023-06-23T18:36:40.492 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:40.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.611+0000 7ff0243e9700 1 Processor -- start 2023-06-23T18:36:40.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.611+0000 7ff0243e9700 1 -- start start 2023-06-23T18:36:40.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.612+0000 7ff0243e9700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff01c156ae0 0x7ff01c15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.612+0000 7ff0243e9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff01c0cc4b0 0x7ff01c071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.612+0000 7ff0243e9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01c15daf0 con 0x7ff01c0cc060 2023-06-23T18:36:40.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.612+0000 7ff0243e9700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01c15a560 con 0x7ff01c156ae0 2023-06-23T18:36:40.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.612+0000 7ff0243e9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c159950 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.612+0000 7ff022185700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff01c156ae0 0x7ff01c15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.612+0000 7ff022986700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff01c0cc4b0 0x7ff01c071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.612+0000 7ff022185700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff01c156ae0 0x7ff01c15d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:47090/0 (socket says 172.21.15.79:47090) 2023-06-23T18:36:40.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.612+0000 7ff021984700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7ff01c0cc060 0x7ff01c071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:52174/0 (socket says 172.21.15.79:52174) 2023-06-23T18:36:40.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.612+0000 7ff022986700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff01c0cc4b0 0x7ff01c071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:33638/0 (socket says 172.21.15.79:33638) 2023-06-23T18:36:40.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.612+0000 7ff022185700 1 -- 172.21.15.79:0/674697145 learned_addr learned my addr 172.21.15.79:0/674697145 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:40.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.612+0000 7ff022986700 1 -- 172.21.15.79:0/674697145 >> v2:172.21.15.155:3300/0 conn(0x7ff01c156ae0 msgr2=0x7ff01c15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:40.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.612+0000 7ff022986700 1 --2- 172.21.15.79:0/674697145 >> v2:172.21.15.155:3300/0 conn(0x7ff01c156ae0 0x7ff01c15d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.613+0000 7ff022986700 1 -- 172.21.15.79:0/674697145 >> v1:172.21.15.155:6789/0 conn(0x7ff01c0cc060 legacy=0x7ff01c071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.613+0000 7ff022986700 1 -- 172.21.15.79:0/674697145 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01c159c30 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.613+0000 7ff021183700 1 -- 172.21.15.79:0/674697145 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349378915 0 0) 0x7ff01c15daf0 con 0x7ff01c0cc060 2023-06-23T18:36:40.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.613+0000 7ff022185700 1 --2- 172.21.15.79:0/674697145 >> v2:172.21.15.155:3300/0 conn(0x7ff01c156ae0 0x7ff01c15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:40.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.613+0000 7ff022986700 1 --2- 172.21.15.79:0/674697145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff01c0cc4b0 0x7ff01c071e20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff014009d90 tx=0x7ff01400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df21febee534e2d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:40.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.613+0000 7ff021183700 1 -- 172.21.15.79:0/674697145 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01400e070 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.613+0000 7ff021183700 1 -- 172.21.15.79:0/674697145 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01400bb20 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.613+0000 7ff021183700 1 -- 172.21.15.79:0/674697145 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014012430 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.613+0000 7ff0243e9700 1 -- 172.21.15.79:0/674697145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff01c0cc4b0 msgr2=0x7ff01c071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.613+0000 7ff0243e9700 1 --2- 172.21.15.79:0/674697145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff01c0cc4b0 0x7ff01c071e20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff014009d90 tx=0x7ff01400b330 comp rx=0 tx=0).stop 2023-06-23T18:36:40.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.614+0000 7ff0243e9700 1 -- 172.21.15.79:0/674697145 shutdown_connections 2023-06-23T18:36:40.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.614+0000 7ff0243e9700 1 --2- 172.21.15.79:0/674697145 >> v2:172.21.15.155:3300/0 conn(0x7ff01c156ae0 0x7ff01c15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.614+0000 7ff0243e9700 1 --2- 172.21.15.79:0/674697145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff01c0cc4b0 0x7ff01c071e20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.614+0000 7ff0243e9700 1 -- 172.21.15.79:0/674697145 >> 172.21.15.79:0/674697145 conn(0x7ff01c06afa0 msgr2=0x7ff01c06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:40.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.614+0000 7ff0243e9700 1 -- 172.21.15.79:0/674697145 shutdown_connections 2023-06-23T18:36:40.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.614+0000 7ff0243e9700 1 -- 172.21.15.79:0/674697145 wait complete. 2023-06-23T18:36:40.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.615+0000 7ff0243e9700 1 Processor -- start 2023-06-23T18:36:40.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.615+0000 7ff0243e9700 1 -- start start 2023-06-23T18:36:40.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.615+0000 7ff0243e9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff01c0cc060 0x7ff01c0afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.615+0000 7ff0243e9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff01c0cc4b0 0x7ff01c154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.615+0000 7ff0243e9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff01c156ae0 0x7ff01c1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff0243e9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c1f1b90 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff0243e9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c1f1e70 con 0x7ff01c156ae0 2023-06-23T18:36:40.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff0243e9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c1f2150 con 0x7ff01c0cc060 2023-06-23T18:36:40.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff021984700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff01c0cc4b0 0x7ff01c154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff021984700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff01c0cc4b0 0x7ff01c154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40874/0 (socket says 172.21.15.79:40874) 2023-06-23T18:36:40.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff021984700 1 -- 172.21.15.79:0/1737152930 learned_addr learned my addr 172.21.15.79:0/1737152930 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:40.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff022986700 1 --2- 172.21.15.79:0/1737152930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff01c156ae0 0x7ff01c1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff022185700 1 --2- 172.21.15.79:0/1737152930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff01c0cc060 0x7ff01c0afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff021984700 1 -- 172.21.15.79:0/1737152930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff01c0cc060 msgr2=0x7ff01c0afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff021984700 1 --2- 172.21.15.79:0/1737152930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff01c0cc060 0x7ff01c0afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff021984700 1 -- 172.21.15.79:0/1737152930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff01c156ae0 msgr2=0x7ff01c1529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff021984700 1 --2- 172.21.15.79:0/1737152930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff01c156ae0 0x7ff01c1529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff021984700 1 -- 172.21.15.79:0/1737152930 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00c009550 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff021984700 1 --2- 172.21.15.79:0/1737152930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff01c0cc4b0 0x7ff01c154a40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff0180011e0 tx=0x7ff01800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:40.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff012ffd700 1 -- 172.21.15.79:0/1737152930 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff018004c80 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff012ffd700 1 -- 172.21.15.79:0/1737152930 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff018004e00 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff0243e9700 1 -- 172.21.15.79:0/1737152930 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff014009a70 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff012ffd700 1 -- 172.21.15.79:0/1737152930 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff018007310 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.616+0000 7ff0243e9700 1 -- 172.21.15.79:0/1737152930 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01c1f2710 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.617+0000 7ff0243e9700 1 -- 172.21.15.79:0/1737152930 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff01c0c6920 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.618+0000 7ff012ffd700 1 -- 172.21.15.79:0/1737152930 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff018007470 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.618+0000 7ff012ffd700 1 --2- 172.21.15.79:0/1737152930 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff00006ace0 0x7ff00006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.618+0000 7ff012ffd700 1 -- 172.21.15.79:0/1737152930 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff018089150 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.618+0000 7ff022185700 1 --2- 172.21.15.79:0/1737152930 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff00006ace0 0x7ff00006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.619+0000 7ff022185700 1 --2- 172.21.15.79:0/1737152930 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff00006ace0 0x7ff00006d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff00c006d30 tx=0x7ff00c0092c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:40.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.622+0000 7ff012ffd700 1 -- 172.21.15.79:0/1737152930 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff018054180 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.810+0000 7ff0243e9700 1 -- 172.21.15.79:0/1737152930 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff01c0672c0 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.811+0000 7ff012ffd700 1 -- 172.21.15.79:0/1737152930 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff0180577a0 con 0x7ff01c0cc4b0 2023-06-23T18:36:40.816 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:40.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.813+0000 7ff0243e9700 1 -- 172.21.15.79:0/1737152930 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff00006ace0 msgr2=0x7ff00006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.813+0000 7ff0243e9700 1 --2- 172.21.15.79:0/1737152930 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff00006ace0 0x7ff00006d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff00c006d30 tx=0x7ff00c0092c0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.813+0000 7ff0243e9700 1 -- 172.21.15.79:0/1737152930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff01c0cc4b0 msgr2=0x7ff01c154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.813+0000 7ff0243e9700 1 --2- 172.21.15.79:0/1737152930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff01c0cc4b0 0x7ff01c154a40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff0180011e0 tx=0x7ff01800ca30 comp rx=0 tx=0).stop 2023-06-23T18:36:40.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.813+0000 7ff0243e9700 1 -- 172.21.15.79:0/1737152930 shutdown_connections 2023-06-23T18:36:40.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.813+0000 7ff0243e9700 1 --2- 172.21.15.79:0/1737152930 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7ff00006ace0 0x7ff00006d190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.813+0000 7ff0243e9700 1 --2- 172.21.15.79:0/1737152930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff01c156ae0 0x7ff01c1529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.813+0000 7ff0243e9700 1 --2- 172.21.15.79:0/1737152930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff01c0cc060 0x7ff01c0afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.813+0000 7ff0243e9700 1 --2- 172.21.15.79:0/1737152930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff01c0cc4b0 0x7ff01c154a40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.813+0000 7ff0243e9700 1 -- 172.21.15.79:0/1737152930 >> 172.21.15.79:0/1737152930 conn(0x7ff01c06afa0 msgr2=0x7ff01c0cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:40.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.813+0000 7ff0243e9700 1 -- 172.21.15.79:0/1737152930 shutdown_connections 2023-06-23T18:36:40.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.813+0000 7ff0243e9700 1 -- 172.21.15.79:0/1737152930 wait complete. 2023-06-23T18:36:40.829 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:40.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.947+0000 7fa91a034700 1 Processor -- start 2023-06-23T18:36:40.953 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.947+0000 7fa91a034700 1 -- start start 2023-06-23T18:36:40.955 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.948+0000 7fa91a034700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa9140cd750 0x7fa914159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.955 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.948+0000 7fa91a034700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa914158db0 0x7fa91415c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.955 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.948+0000 7fa91a034700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91415d3c0 con 0x7fa91415be70 2023-06-23T18:36:40.955 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.948+0000 7fa91a034700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91415c240 con 0x7fa9140cd750 2023-06-23T18:36:40.955 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.948+0000 7fa91a034700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9141591e0 con 0x7fa914158db0 2023-06-23T18:36:40.955 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.948+0000 7fa912ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa9140cd750 0x7fa914159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.948+0000 7fa912ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa9140cd750 0x7fa914159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40884/0 (socket says 172.21.15.79:40884) 2023-06-23T18:36:40.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.948+0000 7fa912ffd700 1 -- 172.21.15.79:0/2104655280 learned_addr learned my addr 172.21.15.79:0/2104655280 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:40.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.949+0000 7fa913fff700 1 --2- 172.21.15.79:0/2104655280 >> v2:172.21.15.155:3300/0 conn(0x7fa914158db0 0x7fa91415c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.949+0000 7fa912ffd700 1 -- 172.21.15.79:0/2104655280 >> v2:172.21.15.155:3300/0 conn(0x7fa914158db0 msgr2=0x7fa91415c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:40.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.949+0000 7fa912ffd700 1 --2- 172.21.15.79:0/2104655280 >> v2:172.21.15.155:3300/0 conn(0x7fa914158db0 0x7fa91415c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.949+0000 7fa912ffd700 1 -- 172.21.15.79:0/2104655280 >> v1:172.21.15.155:6789/0 conn(0x7fa91415be70 legacy=0x7fa9140cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.949+0000 7fa912ffd700 1 -- 172.21.15.79:0/2104655280 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91406a0b0 con 0x7fa9140cd750 2023-06-23T18:36:40.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.949+0000 7fa913fff700 1 --2- 172.21.15.79:0/2104655280 >> v2:172.21.15.155:3300/0 conn(0x7fa914158db0 0x7fa91415c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:40.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.949+0000 7fa912ffd700 1 --2- 172.21.15.79:0/2104655280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9140cd750 0x7fa914159ff0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa9140ccdd0 tx=0x7fa908009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fad8c5ab70036d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:40.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.949+0000 7fa9127fc700 1 -- 172.21.15.79:0/2104655280 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90800b870 con 0x7fa9140cd750 2023-06-23T18:36:40.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.949+0000 7fa9127fc700 1 -- 172.21.15.79:0/2104655280 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90800b9d0 con 0x7fa9140cd750 2023-06-23T18:36:40.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.949+0000 7fa9127fc700 1 -- 172.21.15.79:0/2104655280 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90800bd60 con 0x7fa9140cd750 2023-06-23T18:36:40.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.950+0000 7fa91a034700 1 -- 172.21.15.79:0/2104655280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9140cd750 msgr2=0x7fa914159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.950+0000 7fa91a034700 1 --2- 172.21.15.79:0/2104655280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9140cd750 0x7fa914159ff0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa9140ccdd0 tx=0x7fa908009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.950+0000 7fa91a034700 1 -- 172.21.15.79:0/2104655280 shutdown_connections 2023-06-23T18:36:40.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.950+0000 7fa91a034700 1 --2- 172.21.15.79:0/2104655280 >> v2:172.21.15.155:3300/0 conn(0x7fa914158db0 0x7fa91415c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.950+0000 7fa91a034700 1 --2- 172.21.15.79:0/2104655280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9140cd750 0x7fa914159ff0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.950+0000 7fa91a034700 1 -- 172.21.15.79:0/2104655280 >> 172.21.15.79:0/2104655280 conn(0x7fa9140cfc60 msgr2=0x7fa9140b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:40.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.950+0000 7fa91a034700 1 -- 172.21.15.79:0/2104655280 shutdown_connections 2023-06-23T18:36:40.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.950+0000 7fa91a034700 1 -- 172.21.15.79:0/2104655280 wait complete. 2023-06-23T18:36:40.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.951+0000 7fa91a034700 1 Processor -- start 2023-06-23T18:36:40.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.951+0000 7fa91a034700 1 -- start start 2023-06-23T18:36:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa91a034700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9140cd750 0x7fa9140c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa91a034700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa914158db0 0x7fa91406b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa91a034700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa91415be70 0x7fa9140b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa91a034700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9141f1da0 con 0x7fa914158db0 2023-06-23T18:36:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa91a034700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9141f2080 con 0x7fa91415be70 2023-06-23T18:36:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa91a034700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9141f2360 con 0x7fa9140cd750 2023-06-23T18:36:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa912ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa914158db0 0x7fa91406b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa912ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa914158db0 0x7fa91406b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40892/0 (socket says 172.21.15.79:40892) 2023-06-23T18:36:40.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa912ffd700 1 -- 172.21.15.79:0/3001805583 learned_addr learned my addr 172.21.15.79:0/3001805583 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:40.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa913fff700 1 --2- 172.21.15.79:0/3001805583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa91415be70 0x7fa9140b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa9137fe700 1 --2- 172.21.15.79:0/3001805583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9140cd750 0x7fa9140c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa912ffd700 1 -- 172.21.15.79:0/3001805583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9140cd750 msgr2=0x7fa9140c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa912ffd700 1 --2- 172.21.15.79:0/3001805583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9140cd750 0x7fa9140c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa912ffd700 1 -- 172.21.15.79:0/3001805583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa91415be70 msgr2=0x7fa9140b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:40.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa912ffd700 1 --2- 172.21.15.79:0/3001805583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa91415be70 0x7fa9140b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.952+0000 7fa912ffd700 1 -- 172.21.15.79:0/3001805583 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908009550 con 0x7fa914158db0 2023-06-23T18:36:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.953+0000 7fa912ffd700 1 --2- 172.21.15.79:0/3001805583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa914158db0 0x7fa91406b030 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa908009c10 tx=0x7fa908004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.953+0000 7fa8fbfff700 1 -- 172.21.15.79:0/3001805583 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908004250 con 0x7fa914158db0 2023-06-23T18:36:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.953+0000 7fa8fbfff700 1 -- 172.21.15.79:0/3001805583 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90801f070 con 0x7fa914158db0 2023-06-23T18:36:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.953+0000 7fa91a034700 1 -- 172.21.15.79:0/3001805583 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9141f2640 con 0x7fa914158db0 2023-06-23T18:36:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.953+0000 7fa8fbfff700 1 -- 172.21.15.79:0/3001805583 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9080045b0 con 0x7fa914158db0 2023-06-23T18:36:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.953+0000 7fa91a034700 1 -- 172.21.15.79:0/3001805583 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9141f29b0 con 0x7fa914158db0 2023-06-23T18:36:40.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.954+0000 7fa91a034700 1 -- 172.21.15.79:0/3001805583 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9141f50d0 con 0x7fa914158db0 2023-06-23T18:36:40.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.954+0000 7fa8fbfff700 1 -- 172.21.15.79:0/3001805583 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa908021030 con 0x7fa914158db0 2023-06-23T18:36:40.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.954+0000 7fa8fbfff700 1 --2- 172.21.15.79:0/3001805583 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa90006ace0 0x7fa90006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:40.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.955+0000 7fa8fbfff700 1 -- 172.21.15.79:0/3001805583 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa90808a6b0 con 0x7fa914158db0 2023-06-23T18:36:40.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.955+0000 7fa9137fe700 1 --2- 172.21.15.79:0/3001805583 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa90006ace0 0x7fa90006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:40.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.955+0000 7fa9137fe700 1 --2- 172.21.15.79:0/3001805583 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa90006ace0 0x7fa90006d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa90400f290 tx=0x7fa904009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:40.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:40.959+0000 7fa8fbfff700 1 -- 172.21.15.79:0/3001805583 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa9141f50d0 con 0x7fa914158db0 2023-06-23T18:36:41.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.151+0000 7fa91a034700 1 -- 172.21.15.79:0/3001805583 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9141f50d0 con 0x7fa914158db0 2023-06-23T18:36:41.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.151+0000 7fa8fbfff700 1 -- 172.21.15.79:0/3001805583 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa9141f50d0 con 0x7fa914158db0 2023-06-23T18:36:41.156 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:41.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.153+0000 7fa91a034700 1 -- 172.21.15.79:0/3001805583 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa90006ace0 msgr2=0x7fa90006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:41.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.153+0000 7fa91a034700 1 --2- 172.21.15.79:0/3001805583 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa90006ace0 0x7fa90006d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa90400f290 tx=0x7fa904009040 comp rx=0 tx=0).stop 2023-06-23T18:36:41.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.153+0000 7fa91a034700 1 -- 172.21.15.79:0/3001805583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa914158db0 msgr2=0x7fa91406b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:41.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.153+0000 7fa91a034700 1 --2- 172.21.15.79:0/3001805583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa914158db0 0x7fa91406b030 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa908009c10 tx=0x7fa908004750 comp rx=0 tx=0).stop 2023-06-23T18:36:41.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.153+0000 7fa91a034700 1 -- 172.21.15.79:0/3001805583 shutdown_connections 2023-06-23T18:36:41.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.153+0000 7fa91a034700 1 --2- 172.21.15.79:0/3001805583 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa90006ace0 0x7fa90006d190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.153+0000 7fa91a034700 1 --2- 172.21.15.79:0/3001805583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa91415be70 0x7fa9140b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.153+0000 7fa91a034700 1 --2- 172.21.15.79:0/3001805583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9140cd750 0x7fa9140c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.153+0000 7fa91a034700 1 --2- 172.21.15.79:0/3001805583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa914158db0 0x7fa91406b030 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.153+0000 7fa91a034700 1 -- 172.21.15.79:0/3001805583 >> 172.21.15.79:0/3001805583 conn(0x7fa9140cfc60 msgr2=0x7fa91415aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:41.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.153+0000 7fa91a034700 1 -- 172.21.15.79:0/3001805583 shutdown_connections 2023-06-23T18:36:41.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.153+0000 7fa91a034700 1 -- 172.21.15.79:0/3001805583 wait complete. 2023-06-23T18:36:41.171 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:36:41.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.288+0000 7fefc636e700 1 Processor -- start 2023-06-23T18:36:41.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.288+0000 7fefc636e700 1 -- start start 2023-06-23T18:36:41.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.289+0000 7fefc636e700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fefc0158620 0x7fefc006d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:41.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.289+0000 7fefc636e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefc006b3c0 0x7fefc00ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:41.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.289+0000 7fefc636e700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc006b830 con 0x7fefc00cd510 2023-06-23T18:36:41.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.289+0000 7fefc636e700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc00cd930 con 0x7fefc0158620 2023-06-23T18:36:41.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.289+0000 7fefc636e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc015d130 con 0x7fefc006b3c0 2023-06-23T18:36:41.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.289+0000 7fefbffff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fefc0158620 0x7fefc006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:41.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.289+0000 7fefc490b700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fefc00cd510 0x7fefc006df40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:52210/0 (socket says 172.21.15.79:52210) 2023-06-23T18:36:41.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.289+0000 7fefbf7fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefc006b3c0 0x7fefc00ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:41.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.289+0000 7fefc490b700 1 -- 172.21.15.79:0/4237828671 learned_addr learned my addr 172.21.15.79:0/4237828671 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:41.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.290+0000 7fefbf7fe700 1 -- 172.21.15.79:0/4237828671 >> v2:172.21.15.155:3300/0 conn(0x7fefc0158620 msgr2=0x7fefc006d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:41.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.290+0000 7fefbf7fe700 1 --2- 172.21.15.79:0/4237828671 >> v2:172.21.15.155:3300/0 conn(0x7fefc0158620 0x7fefc006d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.290+0000 7fefbf7fe700 1 -- 172.21.15.79:0/4237828671 >> v1:172.21.15.155:6789/0 conn(0x7fefc00cd510 legacy=0x7fefc006df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:41.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.290+0000 7fefbeffd700 1 -- 172.21.15.79:0/4237828671 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508468887 0 0) 0x7fefb8001490 con 0x7fefc00cd510 2023-06-23T18:36:41.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.290+0000 7fefbf7fe700 1 -- 172.21.15.79:0/4237828671 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc015c370 con 0x7fefc006b3c0 2023-06-23T18:36:41.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.290+0000 7fefbffff700 1 --2- 172.21.15.79:0/4237828671 >> v2:172.21.15.155:3300/0 conn(0x7fefc0158620 0x7fefc006d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:41.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.290+0000 7fefbf7fe700 1 --2- 172.21.15.79:0/4237828671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefc006b3c0 0x7fefc00ccfd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fefb400a9e0 tx=0x7fefb400acf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7858ac482a00773 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:41.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.290+0000 7fefbeffd700 1 -- 172.21.15.79:0/4237828671 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb400cb70 con 0x7fefc006b3c0 2023-06-23T18:36:41.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.290+0000 7fefbeffd700 1 -- 172.21.15.79:0/4237828671 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb400ccd0 con 0x7fefc006b3c0 2023-06-23T18:36:41.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.290+0000 7fefbeffd700 1 -- 172.21.15.79:0/4237828671 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb4018620 con 0x7fefc006b3c0 2023-06-23T18:36:41.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.290+0000 7fefc636e700 1 -- 172.21.15.79:0/4237828671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefc006b3c0 msgr2=0x7fefc00ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:41.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.290+0000 7fefc636e700 1 --2- 172.21.15.79:0/4237828671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefc006b3c0 0x7fefc00ccfd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fefb400a9e0 tx=0x7fefb400acf0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.291+0000 7fefc636e700 1 -- 172.21.15.79:0/4237828671 shutdown_connections 2023-06-23T18:36:41.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.291+0000 7fefc636e700 1 --2- 172.21.15.79:0/4237828671 >> v2:172.21.15.155:3300/0 conn(0x7fefc0158620 0x7fefc006d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.291+0000 7fefc636e700 1 --2- 172.21.15.79:0/4237828671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefc006b3c0 0x7fefc00ccfd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.291+0000 7fefc636e700 1 -- 172.21.15.79:0/4237828671 >> 172.21.15.79:0/4237828671 conn(0x7fefc00b17e0 msgr2=0x7fefc006cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:41.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.291+0000 7fefc636e700 1 -- 172.21.15.79:0/4237828671 shutdown_connections 2023-06-23T18:36:41.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.291+0000 7fefc636e700 1 -- 172.21.15.79:0/4237828671 wait complete. 2023-06-23T18:36:41.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.292+0000 7fefc636e700 1 Processor -- start 2023-06-23T18:36:41.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.292+0000 7fefc636e700 1 -- start start 2023-06-23T18:36:41.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefc636e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefc006b3c0 0x7fefc015b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:41.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefc636e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefc00cd510 0x7fefc00b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:41.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefc636e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefc0158620 0x7fefc00b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:41.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefc636e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc01f5430 con 0x7fefc006b3c0 2023-06-23T18:36:41.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefc636e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc006b140 con 0x7fefc0158620 2023-06-23T18:36:41.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefc636e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc015a2e0 con 0x7fefc00cd510 2023-06-23T18:36:41.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefbffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefc006b3c0 0x7fefc015b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:41.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefbffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefc006b3c0 0x7fefc015b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40902/0 (socket says 172.21.15.79:40902) 2023-06-23T18:36:41.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefbffff700 1 -- 172.21.15.79:0/100171355 learned_addr learned my addr 172.21.15.79:0/100171355 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:41.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefbf7fe700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefc00cd510 0x7fefc00b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:41.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefc490b700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefc0158620 0x7fefc00b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:41.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefbffff700 1 -- 172.21.15.79:0/100171355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefc00cd510 msgr2=0x7fefc00b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:41.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefbffff700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefc00cd510 0x7fefc00b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefbffff700 1 -- 172.21.15.79:0/100171355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefc0158620 msgr2=0x7fefc00b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:41.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefbffff700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefc0158620 0x7fefc00b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefbffff700 1 -- 172.21.15.79:0/100171355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb00094a0 con 0x7fefc006b3c0 2023-06-23T18:36:41.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefc490b700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefc0158620 0x7fefc00b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:41.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefbffff700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefc006b3c0 0x7fefc015b620 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fefb000ebe0 tx=0x7fefb0009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:41.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefbcff9700 1 -- 172.21.15.79:0/100171355 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb000c8b0 con 0x7fefc006b3c0 2023-06-23T18:36:41.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefbcff9700 1 -- 172.21.15.79:0/100171355 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb000ca10 con 0x7fefc006b3c0 2023-06-23T18:36:41.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefc636e700 1 -- 172.21.15.79:0/100171355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb400a640 con 0x7fefc006b3c0 2023-06-23T18:36:41.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefbcff9700 1 -- 172.21.15.79:0/100171355 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb000cda0 con 0x7fefc006b3c0 2023-06-23T18:36:41.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.293+0000 7fefc636e700 1 -- 172.21.15.79:0/100171355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc015a860 con 0x7fefc006b3c0 2023-06-23T18:36:41.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.294+0000 7fefc636e700 1 -- 172.21.15.79:0/100171355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc015b470 con 0x7fefc006b3c0 2023-06-23T18:36:41.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.295+0000 7fefbcff9700 1 -- 172.21.15.79:0/100171355 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fefb0013470 con 0x7fefc006b3c0 2023-06-23T18:36:41.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.295+0000 7fefbcff9700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fefac06ace0 0x7fefac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:41.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.295+0000 7fefbcff9700 1 -- 172.21.15.79:0/100171355 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fefb0089610 con 0x7fefc006b3c0 2023-06-23T18:36:41.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.295+0000 7fefbf7fe700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fefac06ace0 0x7fefac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:41.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.295+0000 7fefbf7fe700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fefac06ace0 0x7fefac06d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fefb400a9e0 tx=0x7fefb401a040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:41.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.299+0000 7fefbcff9700 1 -- 172.21.15.79:0/100171355 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fefb0054680 con 0x7fefc006b3c0 2023-06-23T18:36:41.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.500+0000 7fefc636e700 1 -- 172.21.15.79:0/100171355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fefc006d500 con 0x7fefc006b3c0 2023-06-23T18:36:41.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.500+0000 7fefbcff9700 1 -- 172.21.15.79:0/100171355 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fefb0057ca0 con 0x7fefc006b3c0 2023-06-23T18:36:41.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.502+0000 7fefc636e700 1 -- 172.21.15.79:0/100171355 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fefac06ace0 msgr2=0x7fefac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:41.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.502+0000 7fefc636e700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fefac06ace0 0x7fefac06d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fefb400a9e0 tx=0x7fefb401a040 comp rx=0 tx=0).stop 2023-06-23T18:36:41.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.502+0000 7fefc636e700 1 -- 172.21.15.79:0/100171355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefc006b3c0 msgr2=0x7fefc015b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:41.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.502+0000 7fefc636e700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefc006b3c0 0x7fefc015b620 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fefb000ebe0 tx=0x7fefb0009e20 comp rx=0 tx=0).stop 2023-06-23T18:36:41.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.502+0000 7fefc636e700 1 -- 172.21.15.79:0/100171355 shutdown_connections 2023-06-23T18:36:41.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.502+0000 7fefc636e700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fefac06ace0 0x7fefac06d190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.502+0000 7fefc636e700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefc0158620 0x7fefc00b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.502+0000 7fefc636e700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefc006b3c0 0x7fefc015b620 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.502+0000 7fefc636e700 1 --2- 172.21.15.79:0/100171355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefc00cd510 0x7fefc00b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.502+0000 7fefc636e700 1 -- 172.21.15.79:0/100171355 >> 172.21.15.79:0/100171355 conn(0x7fefc00b17e0 msgr2=0x7fefc00cabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:41.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.503+0000 7fefc636e700 1 -- 172.21.15.79:0/100171355 shutdown_connections 2023-06-23T18:36:41.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.503+0000 7fefc636e700 1 -- 172.21.15.79:0/100171355 wait complete. 2023-06-23T18:36:41.510 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-23T18:36:41.519 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:36:41.520 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:36:41.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:41.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.656+0000 7f8d0a320700 1 Processor -- start 2023-06-23T18:36:41.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.656+0000 7f8d0a320700 1 -- start start 2023-06-23T18:36:41.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d0a320700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8d04159170 0x7f8d040cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:41.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d0a320700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8d040cd900 0x7f8d040ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:41.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d0a320700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040ce620 0x7f8d04071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:41.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d0a320700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d040cdd80 con 0x7f8d04159170 2023-06-23T18:36:41.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d0a320700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d0415cd50 con 0x7f8d040cd900 2023-06-23T18:36:41.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d0a320700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0415cfa0 con 0x7f8d040ce620 2023-06-23T18:36:41.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d03fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8d04159170 0x7f8d040cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:41.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d03fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8d04159170 0x7f8d040cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40910/0 (socket says 172.21.15.79:40910) 2023-06-23T18:36:41.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d03fff700 1 -- 172.21.15.79:0/3903360126 learned_addr learned my addr 172.21.15.79:0/3903360126 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:41.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d037fe700 1 --2- 172.21.15.79:0/3903360126 >> v2:172.21.15.155:3300/0 conn(0x7f8d040cd900 0x7f8d040ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:41.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d088bd700 1 --2- 172.21.15.79:0/3903360126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040ce620 0x7f8d04071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:41.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d03fff700 1 -- 172.21.15.79:0/3903360126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040ce620 msgr2=0x7f8d04071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:41.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d03fff700 1 --2- 172.21.15.79:0/3903360126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040ce620 0x7f8d04071490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d03fff700 1 -- 172.21.15.79:0/3903360126 >> v2:172.21.15.155:3300/0 conn(0x7f8d040cd900 msgr2=0x7f8d040ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:41.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d03fff700 1 --2- 172.21.15.79:0/3903360126 >> v2:172.21.15.155:3300/0 conn(0x7f8d040cd900 0x7f8d040ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.666 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.657+0000 7f8d03fff700 1 -- 172.21.15.79:0/3903360126 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0415d280 con 0x7f8d04159170 2023-06-23T18:36:41.666 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.658+0000 7f8d03fff700 1 --2- 172.21.15.79:0/3903360126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d04159170 0x7f8d040cd3c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8d04070f20 tx=0x7f8cf800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc7604037e3e7da server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:41.666 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.658+0000 7f8d02ffd700 1 -- 172.21.15.79:0/3903360126 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf800dd30 con 0x7f8d04159170 2023-06-23T18:36:41.666 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.658+0000 7f8d02ffd700 1 -- 172.21.15.79:0/3903360126 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf800de90 con 0x7f8d04159170 2023-06-23T18:36:41.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.658+0000 7f8d02ffd700 1 -- 172.21.15.79:0/3903360126 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8010690 con 0x7f8d04159170 2023-06-23T18:36:41.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.658+0000 7f8d0a320700 1 -- 172.21.15.79:0/3903360126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d04159170 msgr2=0x7f8d040cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:41.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.658+0000 7f8d0a320700 1 --2- 172.21.15.79:0/3903360126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d04159170 0x7f8d040cd3c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8d04070f20 tx=0x7f8cf800be70 comp rx=0 tx=0).stop 2023-06-23T18:36:41.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.658+0000 7f8d0a320700 1 -- 172.21.15.79:0/3903360126 shutdown_connections 2023-06-23T18:36:41.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.658+0000 7f8d0a320700 1 --2- 172.21.15.79:0/3903360126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040ce620 0x7f8d04071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.658+0000 7f8d0a320700 1 --2- 172.21.15.79:0/3903360126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d04159170 0x7f8d040cd3c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.658+0000 7f8d0a320700 1 --2- 172.21.15.79:0/3903360126 >> v2:172.21.15.155:3300/0 conn(0x7f8d040cd900 0x7f8d040ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.658+0000 7f8d0a320700 1 -- 172.21.15.79:0/3903360126 >> 172.21.15.79:0/3903360126 conn(0x7f8d0406d660 msgr2=0x7f8d0406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:41.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.658+0000 7f8d0a320700 1 -- 172.21.15.79:0/3903360126 shutdown_connections 2023-06-23T18:36:41.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.658+0000 7f8d0a320700 1 -- 172.21.15.79:0/3903360126 wait complete. 2023-06-23T18:36:41.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.659+0000 7f8d0a320700 1 Processor -- start 2023-06-23T18:36:41.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.660+0000 7f8d0a320700 1 -- start start 2023-06-23T18:36:41.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.660+0000 7f8d0a320700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040cd900 0x7f8d04159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:41.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.660+0000 7f8d0a320700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d040ce620 0x7f8d0414bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:41.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.660+0000 7f8d0a320700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d04159170 0x7f8d040c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:41.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.660+0000 7f8d0a320700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0414c060 con 0x7f8d040ce620 2023-06-23T18:36:41.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.660+0000 7f8d0a320700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d04159bd0 con 0x7f8d04159170 2023-06-23T18:36:41.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.660+0000 7f8d0a320700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d04159eb0 con 0x7f8d040cd900 2023-06-23T18:36:41.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.660+0000 7f8d037fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d040ce620 0x7f8d0414bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:41.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d037fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d040ce620 0x7f8d0414bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40918/0 (socket says 172.21.15.79:40918) 2023-06-23T18:36:41.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d037fe700 1 -- 172.21.15.79:0/3494072303 learned_addr learned my addr 172.21.15.79:0/3494072303 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:41.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d03fff700 1 --2- 172.21.15.79:0/3494072303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040cd900 0x7f8d04159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:41.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d088bd700 1 --2- 172.21.15.79:0/3494072303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d04159170 0x7f8d040c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:41.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d037fe700 1 -- 172.21.15.79:0/3494072303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040cd900 msgr2=0x7f8d04159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:41.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d037fe700 1 --2- 172.21.15.79:0/3494072303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040cd900 0x7f8d04159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d037fe700 1 -- 172.21.15.79:0/3494072303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d04159170 msgr2=0x7f8d040c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:41.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d037fe700 1 --2- 172.21.15.79:0/3494072303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d04159170 0x7f8d040c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d037fe700 1 -- 172.21.15.79:0/3494072303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf800b7f0 con 0x7f8d040ce620 2023-06-23T18:36:41.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d037fe700 1 --2- 172.21.15.79:0/3494072303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d040ce620 0x7f8d0414bb20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400e810 tx=0x7f8cf400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:41.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d00ff9700 1 -- 172.21.15.79:0/3494072303 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf400c070 con 0x7f8d040ce620 2023-06-23T18:36:41.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d00ff9700 1 -- 172.21.15.79:0/3494072303 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf4013610 con 0x7f8d040ce620 2023-06-23T18:36:41.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d00ff9700 1 -- 172.21.15.79:0/3494072303 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf40139c0 con 0x7f8d040ce620 2023-06-23T18:36:41.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d0a320700 1 -- 172.21.15.79:0/3494072303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0415a190 con 0x7f8d040ce620 2023-06-23T18:36:41.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.661+0000 7f8d0a320700 1 -- 172.21.15.79:0/3494072303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0415a750 con 0x7f8d040ce620 2023-06-23T18:36:41.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.662+0000 7f8d0a320700 1 -- 172.21.15.79:0/3494072303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0406d200 con 0x7f8d040ce620 2023-06-23T18:36:41.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.662+0000 7f8d00ff9700 1 -- 172.21.15.79:0/3494072303 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8cf4013b20 con 0x7f8d040ce620 2023-06-23T18:36:41.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.663+0000 7f8d00ff9700 1 --2- 172.21.15.79:0/3494072303 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cf006ad10 0x7f8cf006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:41.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.663+0000 7f8d00ff9700 1 -- 172.21.15.79:0/3494072303 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8cf4089250 con 0x7f8d040ce620 2023-06-23T18:36:41.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.663+0000 7f8d03fff700 1 --2- 172.21.15.79:0/3494072303 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cf006ad10 0x7f8cf006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:41.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.663+0000 7f8d03fff700 1 --2- 172.21.15.79:0/3494072303 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cf006ad10 0x7f8cf006d1c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800beb0 tx=0x7f8cf8012000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:41.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.667+0000 7f8d00ff9700 1 -- 172.21.15.79:0/3494072303 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8cf4054240 con 0x7f8d040ce620 2023-06-23T18:36:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.857+0000 7f8d0a320700 1 -- 172.21.15.79:0/3494072303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d0415a970 con 0x7f8d040ce620 2023-06-23T18:36:41.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.858+0000 7f8d00ff9700 1 -- 172.21.15.79:0/3494072303 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18329 (secure 0 0 0) 0x7f8cf4057860 con 0x7f8d040ce620 2023-06-23T18:36:41.863 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:41.863 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":21,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:36:41.062167+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1a","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:41.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.860+0000 7f8d0a320700 1 -- 172.21.15.79:0/3494072303 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cf006ad10 msgr2=0x7f8cf006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:41.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.860+0000 7f8d0a320700 1 --2- 172.21.15.79:0/3494072303 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cf006ad10 0x7f8cf006d1c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800beb0 tx=0x7f8cf8012000 comp rx=0 tx=0).stop 2023-06-23T18:36:41.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.860+0000 7f8d0a320700 1 -- 172.21.15.79:0/3494072303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d040ce620 msgr2=0x7f8d0414bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:41.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.860+0000 7f8d0a320700 1 --2- 172.21.15.79:0/3494072303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d040ce620 0x7f8d0414bb20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400e810 tx=0x7f8cf400eb20 comp rx=0 tx=0).stop 2023-06-23T18:36:41.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.860+0000 7f8d0a320700 1 -- 172.21.15.79:0/3494072303 shutdown_connections 2023-06-23T18:36:41.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.860+0000 7f8d0a320700 1 --2- 172.21.15.79:0/3494072303 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cf006ad10 0x7f8cf006d1c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.860+0000 7f8d0a320700 1 --2- 172.21.15.79:0/3494072303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d04159170 0x7f8d040c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.860+0000 7f8d0a320700 1 --2- 172.21.15.79:0/3494072303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040cd900 0x7f8d04159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.860+0000 7f8d0a320700 1 --2- 172.21.15.79:0/3494072303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d040ce620 0x7f8d0414bb20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:41.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.860+0000 7f8d0a320700 1 -- 172.21.15.79:0/3494072303 >> 172.21.15.79:0/3494072303 conn(0x7f8d0406d660 msgr2=0x7f8d040af310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:41.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.860+0000 7f8d0a320700 1 -- 172.21.15.79:0/3494072303 shutdown_connections 2023-06-23T18:36:41.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.860+0000 7f8d0a320700 1 -- 172.21.15.79:0/3494072303 wait complete. 2023-06-23T18:36:41.878 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:41.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:42.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.997+0000 7fe3718da700 1 Processor -- start 2023-06-23T18:36:42.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.997+0000 7fe3718da700 1 -- start start 2023-06-23T18:36:42.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.998+0000 7fe3718da700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe36c0b81c0 0x7fe36c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.998+0000 7fe3718da700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe36c0b1a80 0x7fe36c0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.998+0000 7fe3718da700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36c0d1830 con 0x7fe36c0ceda0 2023-06-23T18:36:42.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.998+0000 7fe3718da700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36c0cf330 con 0x7fe36c0b1a80 2023-06-23T18:36:42.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.998+0000 7fe3718da700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36c0d1270 con 0x7fe36c0b81c0 2023-06-23T18:36:42.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.999+0000 7fe36a7fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe36c0b1a80 0x7fe36c0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.999+0000 7fe36a7fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe36c0b1a80 0x7fe36c0cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40926/0 (socket says 172.21.15.79:40926) 2023-06-23T18:36:42.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.999+0000 7fe36b7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe36c0ceda0 0x7fe36c15ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43970/0 (socket says 172.21.15.79:43970) 2023-06-23T18:36:42.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.999+0000 7fe36a7fc700 1 -- 172.21.15.79:0/949715292 learned_addr learned my addr 172.21.15.79:0/949715292 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:42.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.999+0000 7fe36affd700 1 --2- 172.21.15.79:0/949715292 >> v2:172.21.15.155:3300/0 conn(0x7fe36c0b81c0 0x7fe36c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.999+0000 7fe36a7fc700 1 -- 172.21.15.79:0/949715292 >> v2:172.21.15.155:3300/0 conn(0x7fe36c0b81c0 msgr2=0x7fe36c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:42.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.999+0000 7fe36a7fc700 1 --2- 172.21.15.79:0/949715292 >> v2:172.21.15.155:3300/0 conn(0x7fe36c0b81c0 0x7fe36c0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.999+0000 7fe36a7fc700 1 -- 172.21.15.79:0/949715292 >> v1:172.21.15.79:6789/0 conn(0x7fe36c0ceda0 legacy=0x7fe36c15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.999+0000 7fe36a7fc700 1 -- 172.21.15.79:0/949715292 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c0d1550 con 0x7fe36c0b1a80 2023-06-23T18:36:42.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.999+0000 7fe369ffb700 1 -- 172.21.15.79:0/949715292 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204020300 0 0) 0x7fe36c0d1830 con 0x7fe36c0ceda0 2023-06-23T18:36:42.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.999+0000 7fe36a7fc700 1 --2- 172.21.15.79:0/949715292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c0b1a80 0x7fe36c0cfb40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0b2620 tx=0x7fe35c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76dc7e3030510db0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:42.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.999+0000 7fe369ffb700 1 -- 172.21.15.79:0/949715292 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00b870 con 0x7fe36c0b1a80 2023-06-23T18:36:42.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.999+0000 7fe369ffb700 1 -- 172.21.15.79:0/949715292 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c00b9d0 con 0x7fe36c0b1a80 2023-06-23T18:36:42.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:41.999+0000 7fe369ffb700 1 -- 172.21.15.79:0/949715292 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00bd60 con 0x7fe36c0b1a80 2023-06-23T18:36:42.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.000+0000 7fe3718da700 1 -- 172.21.15.79:0/949715292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c0b1a80 msgr2=0x7fe36c0cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.000+0000 7fe3718da700 1 --2- 172.21.15.79:0/949715292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c0b1a80 0x7fe36c0cfb40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0b2620 tx=0x7fe35c009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.000+0000 7fe3718da700 1 -- 172.21.15.79:0/949715292 shutdown_connections 2023-06-23T18:36:42.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.000+0000 7fe3718da700 1 --2- 172.21.15.79:0/949715292 >> v2:172.21.15.155:3300/0 conn(0x7fe36c0b81c0 0x7fe36c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.000+0000 7fe3718da700 1 --2- 172.21.15.79:0/949715292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c0b1a80 0x7fe36c0cfb40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.000+0000 7fe3718da700 1 -- 172.21.15.79:0/949715292 >> 172.21.15.79:0/949715292 conn(0x7fe36c06b2b0 msgr2=0x7fe36c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:42.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.000+0000 7fe3718da700 1 -- 172.21.15.79:0/949715292 shutdown_connections 2023-06-23T18:36:42.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.000+0000 7fe3718da700 1 -- 172.21.15.79:0/949715292 wait complete. 2023-06-23T18:36:42.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.001+0000 7fe3718da700 1 Processor -- start 2023-06-23T18:36:42.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.001+0000 7fe3718da700 1 -- start start 2023-06-23T18:36:42.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe3718da700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c0b1a80 0x7fe36c1f14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe3718da700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe36c0b81c0 0x7fe36c0c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe3718da700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe36c0ceda0 0x7fe36c0c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe3718da700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0c7b80 con 0x7fe36c0b1a80 2023-06-23T18:36:42.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe3718da700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1f1a20 con 0x7fe36c0ceda0 2023-06-23T18:36:42.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe3718da700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0c7000 con 0x7fe36c0b81c0 2023-06-23T18:36:42.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe36affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c0b1a80 0x7fe36c1f14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe36affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c0b1a80 0x7fe36c1f14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40930/0 (socket says 172.21.15.79:40930) 2023-06-23T18:36:42.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe36affd700 1 -- 172.21.15.79:0/3064853130 learned_addr learned my addr 172.21.15.79:0/3064853130 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:42.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe36a7fc700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe36c0b81c0 0x7fe36c0c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe36b7fe700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe36c0ceda0 0x7fe36c0c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe36affd700 1 -- 172.21.15.79:0/3064853130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe36c0b81c0 msgr2=0x7fe36c0c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe36affd700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe36c0b81c0 0x7fe36c0c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe36affd700 1 -- 172.21.15.79:0/3064853130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe36c0ceda0 msgr2=0x7fe36c0c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe36affd700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe36c0ceda0 0x7fe36c0c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe36affd700 1 -- 172.21.15.79:0/3064853130 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c009550 con 0x7fe36c0b1a80 2023-06-23T18:36:42.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe36b7fe700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe36c0ceda0 0x7fe36c0c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:42.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe36a7fc700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe36c0b81c0 0x7fe36c0c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:42.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe36affd700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c0b1a80 0x7fe36c1f14e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe36000e840 tx=0x7fe36000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:42.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.002+0000 7fe353fff700 1 -- 172.21.15.79:0/3064853130 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360013610 con 0x7fe36c0b1a80 2023-06-23T18:36:42.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.003+0000 7fe353fff700 1 -- 172.21.15.79:0/3064853130 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe360013770 con 0x7fe36c0b1a80 2023-06-23T18:36:42.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.003+0000 7fe3718da700 1 -- 172.21.15.79:0/3064853130 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c0c72e0 con 0x7fe36c0b1a80 2023-06-23T18:36:42.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.003+0000 7fe353fff700 1 -- 172.21.15.79:0/3064853130 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360013b20 con 0x7fe36c0b1a80 2023-06-23T18:36:42.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.003+0000 7fe3718da700 1 -- 172.21.15.79:0/3064853130 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c0c78a0 con 0x7fe36c0b1a80 2023-06-23T18:36:42.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.003+0000 7fe3718da700 1 -- 172.21.15.79:0/3064853130 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c04e1d0 con 0x7fe36c0b1a80 2023-06-23T18:36:42.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.004+0000 7fe353fff700 1 -- 172.21.15.79:0/3064853130 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe360013c80 con 0x7fe36c0b1a80 2023-06-23T18:36:42.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.004+0000 7fe353fff700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe35806ace0 0x7fe35806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.004+0000 7fe353fff700 1 -- 172.21.15.79:0/3064853130 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe360089190 con 0x7fe36c0b1a80 2023-06-23T18:36:42.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.004+0000 7fe36a7fc700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe35806ace0 0x7fe35806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.005+0000 7fe36a7fc700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe35806ace0 0x7fe35806d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe36c15fcb0 tx=0x7fe35c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:42.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.009+0000 7fe353fff700 1 -- 172.21.15.79:0/3064853130 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe360054200 con 0x7fe36c0b1a80 2023-06-23T18:36:42.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.196+0000 7fe3718da700 1 -- 172.21.15.79:0/3064853130 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe36c0b1560 con 0x7fe36c0b1a80 2023-06-23T18:36:42.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.196+0000 7fe353fff700 1 -- 172.21.15.79:0/3064853130 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe360057820 con 0x7fe36c0b1a80 2023-06-23T18:36:42.202 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:42.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.198+0000 7fe3718da700 1 -- 172.21.15.79:0/3064853130 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe35806ace0 msgr2=0x7fe35806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.198+0000 7fe3718da700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe35806ace0 0x7fe35806d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe36c15fcb0 tx=0x7fe35c008030 comp rx=0 tx=0).stop 2023-06-23T18:36:42.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.198+0000 7fe3718da700 1 -- 172.21.15.79:0/3064853130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c0b1a80 msgr2=0x7fe36c1f14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.198+0000 7fe3718da700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c0b1a80 0x7fe36c1f14e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe36000e840 tx=0x7fe36000ec00 comp rx=0 tx=0).stop 2023-06-23T18:36:42.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.198+0000 7fe3718da700 1 -- 172.21.15.79:0/3064853130 shutdown_connections 2023-06-23T18:36:42.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.198+0000 7fe3718da700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe35806ace0 0x7fe35806d190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.198+0000 7fe3718da700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe36c0ceda0 0x7fe36c0c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.198+0000 7fe3718da700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c0b1a80 0x7fe36c1f14e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.198+0000 7fe3718da700 1 --2- 172.21.15.79:0/3064853130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe36c0b81c0 0x7fe36c0c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.198+0000 7fe3718da700 1 -- 172.21.15.79:0/3064853130 >> 172.21.15.79:0/3064853130 conn(0x7fe36c06b2b0 msgr2=0x7fe36c0b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:42.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.198+0000 7fe3718da700 1 -- 172.21.15.79:0/3064853130 shutdown_connections 2023-06-23T18:36:42.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.198+0000 7fe3718da700 1 -- 172.21.15.79:0/3064853130 wait complete. 2023-06-23T18:36:42.215 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:42.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.332+0000 7fd25efde700 1 Processor -- start 2023-06-23T18:36:42.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.333+0000 7fd25efde700 1 -- start start 2023-06-23T18:36:42.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.333+0000 7fd25efde700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd258159fb0 0x7fd2580caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.333+0000 7fd25efde700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd2580b2370 0x7fd2580b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.333+0000 7fd25efde700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2580cb480 con 0x7fd25815dbf0 2023-06-23T18:36:42.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.333+0000 7fd25efde700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2580cdc40 con 0x7fd2580b2370 2023-06-23T18:36:42.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.333+0000 7fd25efde700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580b2d20 con 0x7fd258159fb0 2023-06-23T18:36:42.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd25d57b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd25815dbf0 0x7fd25815e030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43984/0 (socket says 172.21.15.79:43984) 2023-06-23T18:36:42.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd25d57b700 1 -- 172.21.15.79:0/3151596215 learned_addr learned my addr 172.21.15.79:0/3151596215 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:42.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 -- 172.21.15.79:0/3151596215 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373785917 0 0) 0x7fd2580cb480 con 0x7fd25815dbf0 2023-06-23T18:36:42.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 -- 172.21.15.79:0/3151596215 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2480035b0 con 0x7fd25815dbf0 2023-06-23T18:36:42.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd25cd7a700 1 --2- 172.21.15.79:0/3151596215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd258159fb0 0x7fd2580caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd257fff700 1 --2- 172.21.15.79:0/3151596215 >> v2:172.21.15.155:3300/0 conn(0x7fd2580b2370 0x7fd2580b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 -- 172.21.15.79:0/3151596215 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1317337420 0 0) 0x7fd2480035b0 con 0x7fd25815dbf0 2023-06-23T18:36:42.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 -- 172.21.15.79:0/3151596215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd258159fb0 msgr2=0x7fd2580caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:42.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 --2- 172.21.15.79:0/3151596215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd258159fb0 0x7fd2580caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 -- 172.21.15.79:0/3151596215 >> v2:172.21.15.155:3300/0 conn(0x7fd2580b2370 msgr2=0x7fd2580b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:42.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 --2- 172.21.15.79:0/3151596215 >> v2:172.21.15.155:3300/0 conn(0x7fd2580b2370 0x7fd2580b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 -- 172.21.15.79:0/3151596215 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2580d0500 con 0x7fd25815dbf0 2023-06-23T18:36:42.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 -- 172.21.15.79:0/3151596215 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fd2500019f0 con 0x7fd25815dbf0 2023-06-23T18:36:42.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 -- 172.21.15.79:0/3151596215 >> v1:172.21.15.79:6789/0 conn(0x7fd25815dbf0 legacy=0x7fd25815e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 --2- 172.21.15.79:0/3151596215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd248003a00 0x7fd248003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 --2- 172.21.15.79:0/3151596215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd248004430 0x7fd2480048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 --2- 172.21.15.79:0/3151596215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd258159fb0 0x7fd248005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 -- 172.21.15.79:0/3151596215 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580cdc40 con 0x7fd248003a00 2023-06-23T18:36:42.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 -- 172.21.15.79:0/3151596215 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580b2d20 con 0x7fd248004430 2023-06-23T18:36:42.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd2577fe700 1 -- 172.21.15.79:0/3151596215 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24800ba10 con 0x7fd258159fb0 2023-06-23T18:36:42.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd25cd7a700 1 --2- 172.21.15.79:0/3151596215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd248003a00 0x7fd248003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd257fff700 1 --2- 172.21.15.79:0/3151596215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd248004430 0x7fd2480048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd25d57b700 1 --2- 172.21.15.79:0/3151596215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd258159fb0 0x7fd248005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd25cd7a700 1 -- 172.21.15.79:0/3151596215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd258159fb0 msgr2=0x7fd248005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:42.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd25cd7a700 1 --2- 172.21.15.79:0/3151596215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd258159fb0 0x7fd248005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd25cd7a700 1 -- 172.21.15.79:0/3151596215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd248004430 msgr2=0x7fd2480048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:42.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd25cd7a700 1 --2- 172.21.15.79:0/3151596215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd248004430 0x7fd2480048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.334+0000 7fd25cd7a700 1 -- 172.21.15.79:0/3151596215 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2480035b0 con 0x7fd248003a00 2023-06-23T18:36:42.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.335+0000 7fd25d57b700 1 --2- 172.21.15.79:0/3151596215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd258159fb0 0x7fd248005060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:42.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.335+0000 7fd25cd7a700 1 --2- 172.21.15.79:0/3151596215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd248003a00 0x7fd248003e10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009b90 tx=0x7fd248005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebe7b7bfc40d917 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:42.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.335+0000 7fd2577fe700 1 -- 172.21.15.79:0/3151596215 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c00e6c0 con 0x7fd248003a00 2023-06-23T18:36:42.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.335+0000 7fd2577fe700 1 -- 172.21.15.79:0/3151596215 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c00e820 con 0x7fd248003a00 2023-06-23T18:36:42.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.335+0000 7fd2577fe700 1 -- 172.21.15.79:0/3151596215 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c00eb90 con 0x7fd248003a00 2023-06-23T18:36:42.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.335+0000 7fd25efde700 1 -- 172.21.15.79:0/3151596215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd248003a00 msgr2=0x7fd248003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.335+0000 7fd25efde700 1 --2- 172.21.15.79:0/3151596215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd248003a00 0x7fd248003e10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009b90 tx=0x7fd248005880 comp rx=0 tx=0).stop 2023-06-23T18:36:42.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.335+0000 7fd25d57b700 1 -- 172.21.15.79:0/3151596215 reap_dead start 2023-06-23T18:36:42.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.335+0000 7fd25efde700 1 -- 172.21.15.79:0/3151596215 shutdown_connections 2023-06-23T18:36:42.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.335+0000 7fd25efde700 1 -- 172.21.15.79:0/3151596215 >> 172.21.15.79:0/3151596215 conn(0x7fd25806e540 msgr2=0x7fd25806e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:42.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.335+0000 7fd25efde700 1 -- 172.21.15.79:0/3151596215 shutdown_connections 2023-06-23T18:36:42.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.335+0000 7fd25efde700 1 -- 172.21.15.79:0/3151596215 wait complete. 2023-06-23T18:36:42.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.336+0000 7fd25efde700 1 Processor -- start 2023-06-23T18:36:42.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.337+0000 7fd25efde700 1 -- start start 2023-06-23T18:36:42.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.337+0000 7fd25efde700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd248003a00 0x7fd25814ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.337+0000 7fd25efde700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2580c9f80 0x7fd2580c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.337+0000 7fd25efde700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2580c6880 0x7fd2580c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.337+0000 7fd25efde700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2581f0a90 con 0x7fd248003a00 2023-06-23T18:36:42.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.337+0000 7fd25efde700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25814b2e0 con 0x7fd2580c6880 2023-06-23T18:36:42.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.337+0000 7fd25efde700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580c6cf0 con 0x7fd2580c9f80 2023-06-23T18:36:42.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.337+0000 7fd25cd7a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd248003a00 0x7fd25814ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd25cd7a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd248003a00 0x7fd25814ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40950/0 (socket says 172.21.15.79:40950) 2023-06-23T18:36:42.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd25cd7a700 1 -- 172.21.15.79:0/374703593 learned_addr learned my addr 172.21.15.79:0/374703593 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:42.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd257fff700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2580c9f80 0x7fd2580c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd25d57b700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2580c6880 0x7fd2580c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd25cd7a700 1 -- 172.21.15.79:0/374703593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2580c9f80 msgr2=0x7fd2580c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd25cd7a700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2580c9f80 0x7fd2580c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd25cd7a700 1 -- 172.21.15.79:0/374703593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2580c6880 msgr2=0x7fd2580c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd25cd7a700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2580c6880 0x7fd2580c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd25cd7a700 1 -- 172.21.15.79:0/374703593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2581efb50 con 0x7fd248003a00 2023-06-23T18:36:42.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd257fff700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2580c9f80 0x7fd2580c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:42.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd25cd7a700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd248003a00 0x7fd25814ada0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd248005770 tx=0x7fd24c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:42.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd2557fa700 1 -- 172.21.15.79:0/374703593 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c00e6c0 con 0x7fd248003a00 2023-06-23T18:36:42.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd2557fa700 1 -- 172.21.15.79:0/374703593 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c003ab0 con 0x7fd248003a00 2023-06-23T18:36:42.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd2557fa700 1 -- 172.21.15.79:0/374703593 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c003e60 con 0x7fd248003a00 2023-06-23T18:36:42.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd25efde700 1 -- 172.21.15.79:0/374703593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2581efe10 con 0x7fd248003a00 2023-06-23T18:36:42.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.338+0000 7fd25efde700 1 -- 172.21.15.79:0/374703593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2581f03d0 con 0x7fd248003a00 2023-06-23T18:36:42.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.339+0000 7fd25efde700 1 -- 172.21.15.79:0/374703593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25815a650 con 0x7fd248003a00 2023-06-23T18:36:42.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.339+0000 7fd2557fa700 1 -- 172.21.15.79:0/374703593 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd24c013470 con 0x7fd248003a00 2023-06-23T18:36:42.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.340+0000 7fd2557fa700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd23c06ace0 0x7fd23c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.340+0000 7fd2557fa700 1 -- 172.21.15.79:0/374703593 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd24c08ae70 con 0x7fd248003a00 2023-06-23T18:36:42.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.340+0000 7fd257fff700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd23c06ace0 0x7fd23c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.340+0000 7fd257fff700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd23c06ace0 0x7fd23c06d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd2580cab30 tx=0x7fd244008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:42.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.344+0000 7fd2557fa700 1 -- 172.21.15.79:0/374703593 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd24c055f70 con 0x7fd248003a00 2023-06-23T18:36:42.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.532+0000 7fd25efde700 1 -- 172.21.15.79:0/374703593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd25804e1d0 con 0x7fd248003a00 2023-06-23T18:36:42.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.533+0000 7fd2557fa700 1 -- 172.21.15.79:0/374703593 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd24c059590 con 0x7fd248003a00 2023-06-23T18:36:42.539 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.535+0000 7fd25efde700 1 -- 172.21.15.79:0/374703593 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd23c06ace0 msgr2=0x7fd23c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.535+0000 7fd25efde700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd23c06ace0 0x7fd23c06d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd2580cab30 tx=0x7fd244008040 comp rx=0 tx=0).stop 2023-06-23T18:36:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.535+0000 7fd25efde700 1 -- 172.21.15.79:0/374703593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd248003a00 msgr2=0x7fd25814ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.535+0000 7fd25efde700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd248003a00 0x7fd25814ada0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd248005770 tx=0x7fd24c0047c0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.535+0000 7fd25efde700 1 -- 172.21.15.79:0/374703593 shutdown_connections 2023-06-23T18:36:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.535+0000 7fd25efde700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd23c06ace0 0x7fd23c06d190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.535+0000 7fd25efde700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2580c6880 0x7fd2580c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.535+0000 7fd25efde700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd248003a00 0x7fd25814ada0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.535+0000 7fd25efde700 1 --2- 172.21.15.79:0/374703593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2580c9f80 0x7fd2580c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.535+0000 7fd25efde700 1 -- 172.21.15.79:0/374703593 >> 172.21.15.79:0/374703593 conn(0x7fd25806e540 msgr2=0x7fd2580caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:42.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.535+0000 7fd25efde700 1 -- 172.21.15.79:0/374703593 shutdown_connections 2023-06-23T18:36:42.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.535+0000 7fd25efde700 1 -- 172.21.15.79:0/374703593 wait complete. 2023-06-23T18:36:42.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:42.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.670+0000 7f8ea9572700 1 Processor -- start 2023-06-23T18:36:42.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.670+0000 7f8ea9572700 1 -- start start 2023-06-23T18:36:42.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.670+0000 7f8ea9572700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8ea40cfd10 0x7f8ea40d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea9572700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea40d1830 con 0x7f8ea40b81c0 2023-06-23T18:36:42.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea9572700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea40cf330 con 0x7f8ea40b1a80 2023-06-23T18:36:42.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea9572700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea40d1270 con 0x7f8ea40cfd10 2023-06-23T18:36:42.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea2ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f8ea40b81c0 0x7f8ea40cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44000/0 (socket says 172.21.15.79:44000) 2023-06-23T18:36:42.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea2ffd700 1 -- 172.21.15.79:0/3339889975 learned_addr learned my addr 172.21.15.79:0/3339889975 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:42.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 -- 172.21.15.79:0/3339889975 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795006347 0 0) 0x7f8ea40d1830 con 0x7f8ea40b81c0 2023-06-23T18:36:42.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 -- 172.21.15.79:0/3339889975 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e8c0035b0 con 0x7f8ea40b81c0 2023-06-23T18:36:42.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea27fc700 1 --2- 172.21.15.79:0/3339889975 >> v2:172.21.15.155:3300/0 conn(0x7f8ea40cfd10 0x7f8ea40d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 -- 172.21.15.79:0/3339889975 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2584536698 0 0) 0x7f8e8c0035b0 con 0x7f8ea40b81c0 2023-06-23T18:36:42.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 -- 172.21.15.79:0/3339889975 >> v2:172.21.15.155:3300/0 conn(0x7f8ea40cfd10 msgr2=0x7f8ea40d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:42.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 --2- 172.21.15.79:0/3339889975 >> v2:172.21.15.155:3300/0 conn(0x7f8ea40cfd10 0x7f8ea40d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 -- 172.21.15.79:0/3339889975 >> v1:172.21.15.155:6789/0 conn(0x7f8ea40b1a80 legacy=0x7f8ea40d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 -- 172.21.15.79:0/3339889975 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea40d1550 con 0x7f8ea40b81c0 2023-06-23T18:36:42.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 -- 172.21.15.79:0/3339889975 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f8e94001670 con 0x7f8ea40b81c0 2023-06-23T18:36:42.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 -- 172.21.15.79:0/3339889975 >> v1:172.21.15.79:6789/0 conn(0x7f8ea40b81c0 legacy=0x7f8ea40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 --2- 172.21.15.79:0/3339889975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e8c003a00 0x7f8e8c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 --2- 172.21.15.79:0/3339889975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e8c004850 0x7f8e8c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 --2- 172.21.15.79:0/3339889975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e8c005200 0x7f8e8c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 -- 172.21.15.79:0/3339889975 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40d1270 con 0x7f8e8c004850 2023-06-23T18:36:42.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 -- 172.21.15.79:0/3339889975 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c00c080 con 0x7f8e8c005200 2023-06-23T18:36:42.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea1ffb700 1 -- 172.21.15.79:0/3339889975 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c00c240 con 0x7f8e8c003a00 2023-06-23T18:36:42.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea27fc700 1 --2- 172.21.15.79:0/3339889975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e8c004850 0x7f8e8c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.671+0000 7f8ea2ffd700 1 --2- 172.21.15.79:0/3339889975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e8c003a00 0x7f8e8c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea37fe700 1 --2- 172.21.15.79:0/3339889975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e8c005200 0x7f8e8c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea27fc700 1 -- 172.21.15.79:0/3339889975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e8c003a00 msgr2=0x7f8e8c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:42.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea27fc700 1 --2- 172.21.15.79:0/3339889975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e8c003a00 0x7f8e8c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea27fc700 1 -- 172.21.15.79:0/3339889975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e8c005200 msgr2=0x7f8e8c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:42.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea27fc700 1 --2- 172.21.15.79:0/3339889975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e8c005200 0x7f8e8c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea37fe700 1 -- 172.21.15.79:0/3339889975 reap_dead start 2023-06-23T18:36:42.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea27fc700 1 -- 172.21.15.79:0/3339889975 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c0035b0 con 0x7f8e8c004850 2023-06-23T18:36:42.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea27fc700 1 --2- 172.21.15.79:0/3339889975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e8c004850 0x7f8e8c004cc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009b30 tx=0x7f8e8c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff707f8267cd9c6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:42.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea1ffb700 1 -- 172.21.15.79:0/3339889975 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800e6c0 con 0x7f8e8c004850 2023-06-23T18:36:42.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea1ffb700 1 -- 172.21.15.79:0/3339889975 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9800e820 con 0x7f8e8c004850 2023-06-23T18:36:42.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea1ffb700 1 -- 172.21.15.79:0/3339889975 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800eb90 con 0x7f8e8c004850 2023-06-23T18:36:42.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea9572700 1 -- 172.21.15.79:0/3339889975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e8c004850 msgr2=0x7f8e8c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea9572700 1 --2- 172.21.15.79:0/3339889975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e8c004850 0x7f8e8c004cc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009b30 tx=0x7f8e8c005880 comp rx=0 tx=0).stop 2023-06-23T18:36:42.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea9572700 1 -- 172.21.15.79:0/3339889975 shutdown_connections 2023-06-23T18:36:42.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea9572700 1 --2- 172.21.15.79:0/3339889975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e8c004850 0x7f8e8c004cc0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea9572700 1 -- 172.21.15.79:0/3339889975 >> 172.21.15.79:0/3339889975 conn(0x7f8ea406b2b0 msgr2=0x7f8ea406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:42.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea9572700 1 -- 172.21.15.79:0/3339889975 shutdown_connections 2023-06-23T18:36:42.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.672+0000 7f8ea9572700 1 -- 172.21.15.79:0/3339889975 wait complete. 2023-06-23T18:36:42.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.674+0000 7f8ea9572700 1 Processor -- start 2023-06-23T18:36:42.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.674+0000 7f8ea9572700 1 -- start start 2023-06-23T18:36:42.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.674+0000 7f8ea9572700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e8c004850 0x7f8ea415ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea9572700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ea414d3e0 0x7f8ea40b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea9572700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea40b3e30 0x7f8ea41f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea9572700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40b42d0 con 0x7f8e8c004850 2023-06-23T18:36:42.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea9572700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40c76a0 con 0x7f8ea414d3e0 2023-06-23T18:36:42.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea9572700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40c6a90 con 0x7f8ea40b3e30 2023-06-23T18:36:42.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea2ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e8c004850 0x7f8ea415ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea2ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e8c004850 0x7f8ea415ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40970/0 (socket says 172.21.15.79:40970) 2023-06-23T18:36:42.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea2ffd700 1 -- 172.21.15.79:0/3363284345 learned_addr learned my addr 172.21.15.79:0/3363284345 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:42.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea37fe700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea40b3e30 0x7f8ea41f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea27fc700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ea414d3e0 0x7f8ea40b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea2ffd700 1 -- 172.21.15.79:0/3363284345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea40b3e30 msgr2=0x7f8ea41f5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea2ffd700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea40b3e30 0x7f8ea41f5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea2ffd700 1 -- 172.21.15.79:0/3363284345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ea414d3e0 msgr2=0x7f8ea40b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea2ffd700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ea414d3e0 0x7f8ea40b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea2ffd700 1 -- 172.21.15.79:0/3363284345 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea40c6d70 con 0x7f8e8c004850 2023-06-23T18:36:42.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea37fe700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea40b3e30 0x7f8ea41f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:42.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea27fc700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ea414d3e0 0x7f8ea40b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:42.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea2ffd700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e8c004850 0x7f8ea415ca20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8e94001200 tx=0x7f8e9400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:42.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8e8b7fe700 1 -- 172.21.15.79:0/3363284345 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9400c5f0 con 0x7f8e8c004850 2023-06-23T18:36:42.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8e8b7fe700 1 -- 172.21.15.79:0/3363284345 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9400c750 con 0x7f8e8c004850 2023-06-23T18:36:42.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea9572700 1 -- 172.21.15.79:0/3363284345 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea40c7050 con 0x7f8e8c004850 2023-06-23T18:36:42.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8e8b7fe700 1 -- 172.21.15.79:0/3363284345 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9400cb00 con 0x7f8e8c004850 2023-06-23T18:36:42.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.675+0000 7f8ea9572700 1 -- 172.21.15.79:0/3363284345 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea415cf60 con 0x7f8e8c004850 2023-06-23T18:36:42.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.676+0000 7f8ea9572700 1 -- 172.21.15.79:0/3363284345 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea404e1d0 con 0x7f8e8c004850 2023-06-23T18:36:42.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.677+0000 7f8e8b7fe700 1 -- 172.21.15.79:0/3363284345 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8e9400cc60 con 0x7f8e8c004850 2023-06-23T18:36:42.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.677+0000 7f8e8b7fe700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8e9006ace0 0x7f8e9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:42.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.677+0000 7f8e8b7fe700 1 -- 172.21.15.79:0/3363284345 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8e94089370 con 0x7f8e8c004850 2023-06-23T18:36:42.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.677+0000 7f8ea27fc700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8e9006ace0 0x7f8e9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:42.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.677+0000 7f8ea27fc700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8e9006ace0 0x7f8e9006d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8ea406bbd0 tx=0x7f8e98008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:42.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.681+0000 7f8e8b7fe700 1 -- 172.21.15.79:0/3363284345 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8e940543e0 con 0x7f8e8c004850 2023-06-23T18:36:42.873 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.868+0000 7f8ea9572700 1 -- 172.21.15.79:0/3363284345 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ea40b4f30 con 0x7f8e8c004850 2023-06-23T18:36:42.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.869+0000 7f8e8b7fe700 1 -- 172.21.15.79:0/3363284345 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8e94057a00 con 0x7f8e8c004850 2023-06-23T18:36:42.874 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:42.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.871+0000 7f8ea9572700 1 -- 172.21.15.79:0/3363284345 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8e9006ace0 msgr2=0x7f8e9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.871+0000 7f8ea9572700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8e9006ace0 0x7f8e9006d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8ea406bbd0 tx=0x7f8e98008030 comp rx=0 tx=0).stop 2023-06-23T18:36:42.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.871+0000 7f8ea9572700 1 -- 172.21.15.79:0/3363284345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e8c004850 msgr2=0x7f8ea415ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:42.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.871+0000 7f8ea9572700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e8c004850 0x7f8ea415ca20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8e94001200 tx=0x7f8e9400ebd0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.871+0000 7f8ea9572700 1 -- 172.21.15.79:0/3363284345 shutdown_connections 2023-06-23T18:36:42.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.871+0000 7f8ea9572700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea40b3e30 0x7f8ea41f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.871+0000 7f8ea9572700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e8c004850 0x7f8ea415ca20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.871+0000 7f8ea9572700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8e9006ace0 0x7f8e9006d190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.871+0000 7f8ea9572700 1 --2- 172.21.15.79:0/3363284345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ea414d3e0 0x7f8ea40b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:42.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.871+0000 7f8ea9572700 1 -- 172.21.15.79:0/3363284345 >> 172.21.15.79:0/3363284345 conn(0x7f8ea406b2b0 msgr2=0x7f8ea40b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:42.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.871+0000 7f8ea9572700 1 -- 172.21.15.79:0/3363284345 shutdown_connections 2023-06-23T18:36:42.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:42.871+0000 7f8ea9572700 1 -- 172.21.15.79:0/3363284345 wait complete. 2023-06-23T18:36:42.888 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.006+0000 7f510ba41700 1 Processor -- start 2023-06-23T18:36:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.006+0000 7f510ba41700 1 -- start start 2023-06-23T18:36:43.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f510ba41700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51040cb890 0x7f51040cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f510ba41700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51040cc9f0 con 0x7f5104156ae0 2023-06-23T18:36:43.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f510ba41700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f510415a560 con 0x7f51040cc060 2023-06-23T18:36:43.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f510ba41700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51040cbd10 con 0x7f51040cb890 2023-06-23T18:36:43.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f51097dd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5104156ae0 0x7f510415d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44020/0 (socket says 172.21.15.79:44020) 2023-06-23T18:36:43.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f51097dd700 1 -- 172.21.15.79:0/4104868413 learned_addr learned my addr 172.21.15.79:0/4104868413 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:43.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f5109fde700 1 --2- 172.21.15.79:0/4104868413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51040cb890 0x7f51040cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449403436 0 0) 0x7f5100001110 con 0x7f5104156ae0 2023-06-23T18:36:43.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50f00035b0 con 0x7f5104156ae0 2023-06-23T18:36:43.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075523791 0 0) 0x7f510415a560 con 0x7f51040cc060 2023-06-23T18:36:43.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5100001110 con 0x7f51040cc060 2023-06-23T18:36:43.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 91804744 0 0) 0x7f50f00035b0 con 0x7f5104156ae0 2023-06-23T18:36:43.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51040cb890 msgr2=0x7f51040cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:43.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f50fbfff700 1 --2- 172.21.15.79:0/4104868413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51040cb890 0x7f51040cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 >> v1:172.21.15.155:6789/0 conn(0x7f51040cc060 legacy=0x7f5104071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5104159950 con 0x7f5104156ae0 2023-06-23T18:36:43.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f5100001800 con 0x7f5104156ae0 2023-06-23T18:36:43.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 >> v1:172.21.15.79:6789/0 conn(0x7f5104156ae0 legacy=0x7f510415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f50fbfff700 1 --2- 172.21.15.79:0/4104868413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f0003bc0 0x7f50f0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f50fbfff700 1 --2- 172.21.15.79:0/4104868413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50f0004850 0x7f50f0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.007+0000 7f50fbfff700 1 --2- 172.21.15.79:0/4104868413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50f0005200 0x7f50f000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51040cbd10 con 0x7f50f0005200 2023-06-23T18:36:43.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f000bfd0 con 0x7f50f0004850 2023-06-23T18:36:43.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f000c190 con 0x7f50f0003bc0 2023-06-23T18:36:43.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f5109fde700 1 --2- 172.21.15.79:0/4104868413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51040cb890 0x7f51040cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:43.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f5109fde700 1 --2- 172.21.15.79:0/4104868413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50f0005200 0x7f50f000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f5108fdc700 1 --2- 172.21.15.79:0/4104868413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50f0004850 0x7f50f0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f51097dd700 1 --2- 172.21.15.79:0/4104868413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f0003bc0 0x7f50f0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f5109fde700 1 -- 172.21.15.79:0/4104868413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f0003bc0 msgr2=0x7f50f0003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:43.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f5109fde700 1 --2- 172.21.15.79:0/4104868413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f0003bc0 0x7f50f0003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f5109fde700 1 -- 172.21.15.79:0/4104868413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50f0004850 msgr2=0x7f50f0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:43.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f5109fde700 1 --2- 172.21.15.79:0/4104868413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50f0004850 0x7f50f0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f5109fde700 1 -- 172.21.15.79:0/4104868413 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f00035b0 con 0x7f50f0005200 2023-06-23T18:36:43.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f5109fde700 1 --2- 172.21.15.79:0/4104868413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50f0005200 0x7f50f000ba10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f51040cbc60 tx=0x7f50f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca57c070e4dad3b5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:43.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc003c20 con 0x7f50f0005200 2023-06-23T18:36:43.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc00f040 con 0x7f50f0005200 2023-06-23T18:36:43.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.008+0000 7f50fbfff700 1 -- 172.21.15.79:0/4104868413 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc00ea60 con 0x7f50f0005200 2023-06-23T18:36:43.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.009+0000 7f510ba41700 1 -- 172.21.15.79:0/4104868413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50f0005200 msgr2=0x7f50f000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.009+0000 7f510ba41700 1 --2- 172.21.15.79:0/4104868413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50f0005200 0x7f50f000ba10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f51040cbc60 tx=0x7f50f0005880 comp rx=0 tx=0).stop 2023-06-23T18:36:43.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.009+0000 7f5109fde700 1 -- 172.21.15.79:0/4104868413 reap_dead start 2023-06-23T18:36:43.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.009+0000 7f510ba41700 1 -- 172.21.15.79:0/4104868413 shutdown_connections 2023-06-23T18:36:43.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.009+0000 7f510ba41700 1 -- 172.21.15.79:0/4104868413 >> 172.21.15.79:0/4104868413 conn(0x7f510406afa0 msgr2=0x7f510406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:43.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.009+0000 7f510ba41700 1 -- 172.21.15.79:0/4104868413 shutdown_connections 2023-06-23T18:36:43.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.009+0000 7f510ba41700 1 -- 172.21.15.79:0/4104868413 wait complete. 2023-06-23T18:36:43.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.010+0000 7f510ba41700 1 Processor -- start 2023-06-23T18:36:43.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.010+0000 7f510ba41700 1 -- start start 2023-06-23T18:36:43.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f510ba41700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51041f1640 0x7f51041f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f510ba41700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51040c7840 0x7f51040c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f510ba41700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f510414c3f0 0x7f510414c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f510ba41700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51040c8a60 con 0x7f510414c3f0 2023-06-23T18:36:43.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f510ba41700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51041f1f50 con 0x7f51041f1640 2023-06-23T18:36:43.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f510ba41700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51040c7de0 con 0x7f51040c7840 2023-06-23T18:36:43.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f5109fde700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f510414c3f0 0x7f510414c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f5109fde700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f510414c3f0 0x7f510414c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40994/0 (socket says 172.21.15.79:40994) 2023-06-23T18:36:43.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f5109fde700 1 -- 172.21.15.79:0/3090228737 learned_addr learned my addr 172.21.15.79:0/3090228737 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:43.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f51097dd700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51041f1640 0x7f51041f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f5108fdc700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51040c7840 0x7f51040c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f5109fde700 1 -- 172.21.15.79:0/3090228737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51040c7840 msgr2=0x7f51040c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f5109fde700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51040c7840 0x7f51040c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f5109fde700 1 -- 172.21.15.79:0/3090228737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51041f1640 msgr2=0x7f51041f1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f5109fde700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51041f1640 0x7f51041f1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f5109fde700 1 -- 172.21.15.79:0/3090228737 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51040c80c0 con 0x7f510414c3f0 2023-06-23T18:36:43.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f5108fdc700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51040c7840 0x7f51040c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:43.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f5109fde700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f510414c3f0 0x7f510414c860 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f50fc016040 tx=0x7f50fc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:43.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.011+0000 7f50f9ffb700 1 -- 172.21.15.79:0/3090228737 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc00ee50 con 0x7f510414c3f0 2023-06-23T18:36:43.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.012+0000 7f50f9ffb700 1 -- 172.21.15.79:0/3090228737 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc003a20 con 0x7f510414c3f0 2023-06-23T18:36:43.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.012+0000 7f510ba41700 1 -- 172.21.15.79:0/3090228737 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51040c83a0 con 0x7f510414c3f0 2023-06-23T18:36:43.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.012+0000 7f50f9ffb700 1 -- 172.21.15.79:0/3090228737 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc003dd0 con 0x7f510414c3f0 2023-06-23T18:36:43.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.012+0000 7f510ba41700 1 -- 172.21.15.79:0/3090228737 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51040c9d90 con 0x7f510414c3f0 2023-06-23T18:36:43.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.013+0000 7f510ba41700 1 -- 172.21.15.79:0/3090228737 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51040672c0 con 0x7f510414c3f0 2023-06-23T18:36:43.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.013+0000 7f50f9ffb700 1 -- 172.21.15.79:0/3090228737 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f50fc024070 con 0x7f510414c3f0 2023-06-23T18:36:43.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.013+0000 7f50f9ffb700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f50e806ad10 0x7f50e806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.014+0000 7f50f9ffb700 1 -- 172.21.15.79:0/3090228737 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f50fc08ae90 con 0x7f510414c3f0 2023-06-23T18:36:43.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.014+0000 7f51097dd700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f50e806ad10 0x7f50e806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.014+0000 7f51097dd700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f50e806ad10 0x7f50e806d1c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5100000c00 tx=0x7f5100007a30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:43.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.018+0000 7f50f9ffb700 1 -- 172.21.15.79:0/3090228737 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f50fc0560b0 con 0x7f510414c3f0 2023-06-23T18:36:43.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.206+0000 7f510ba41700 1 -- 172.21.15.79:0/3090228737 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f51040b1be0 con 0x7f510414c3f0 2023-06-23T18:36:43.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.206+0000 7f50f9ffb700 1 -- 172.21.15.79:0/3090228737 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f50fc0596d0 con 0x7f510414c3f0 2023-06-23T18:36:43.211 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:43.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.208+0000 7f510ba41700 1 -- 172.21.15.79:0/3090228737 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f50e806ad10 msgr2=0x7f50e806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.208+0000 7f510ba41700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f50e806ad10 0x7f50e806d1c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5100000c00 tx=0x7f5100007a30 comp rx=0 tx=0).stop 2023-06-23T18:36:43.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.208+0000 7f510ba41700 1 -- 172.21.15.79:0/3090228737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f510414c3f0 msgr2=0x7f510414c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.208+0000 7f510ba41700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f510414c3f0 0x7f510414c860 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f50fc016040 tx=0x7f50fc009750 comp rx=0 tx=0).stop 2023-06-23T18:36:43.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.208+0000 7f510ba41700 1 -- 172.21.15.79:0/3090228737 shutdown_connections 2023-06-23T18:36:43.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.208+0000 7f510ba41700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f510414c3f0 0x7f510414c860 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.208+0000 7f510ba41700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f50e806ad10 0x7f50e806d1c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.208+0000 7f510ba41700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51041f1640 0x7f51041f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.208+0000 7f510ba41700 1 --2- 172.21.15.79:0/3090228737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51040c7840 0x7f51040c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.208+0000 7f510ba41700 1 -- 172.21.15.79:0/3090228737 >> 172.21.15.79:0/3090228737 conn(0x7f510406afa0 msgr2=0x7f51041590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:43.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.208+0000 7f510ba41700 1 -- 172.21.15.79:0/3090228737 shutdown_connections 2023-06-23T18:36:43.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.208+0000 7f510ba41700 1 -- 172.21.15.79:0/3090228737 wait complete. 2023-06-23T18:36:43.225 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:36:43.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.340+0000 7f689b5c7700 1 Processor -- start 2023-06-23T18:36:43.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.340+0000 7f689b5c7700 1 -- start start 2023-06-23T18:36:43.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.341+0000 7f689b5c7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6894159170 0x7f68940cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.341+0000 7f689b5c7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68940cd900 0x7f68940ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.341+0000 7f689b5c7700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689415cd50 con 0x7f68940ce620 2023-06-23T18:36:43.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.341+0000 7f689b5c7700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68940cd1e0 con 0x7f68940cd900 2023-06-23T18:36:43.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.341+0000 7f689b5c7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68940cdd70 con 0x7f6894159170 2023-06-23T18:36:43.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.341+0000 7f6898b62700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68940cd900 0x7f68940ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.341+0000 7f6898b62700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68940cd900 0x7f68940ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40998/0 (socket says 172.21.15.79:40998) 2023-06-23T18:36:43.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.341+0000 7f6898b62700 1 -- 172.21.15.79:0/3453157710 learned_addr learned my addr 172.21.15.79:0/3453157710 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:43.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.341+0000 7f6899363700 1 --2- 172.21.15.79:0/3453157710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6894159170 0x7f68940cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f6898b62700 1 -- 172.21.15.79:0/3453157710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6894159170 msgr2=0x7f68940cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:43.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f6898b62700 1 --2- 172.21.15.79:0/3453157710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6894159170 0x7f68940cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f6898b62700 1 -- 172.21.15.79:0/3453157710 >> v1:172.21.15.155:6789/0 conn(0x7f68940ce620 legacy=0x7f6894071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f6898b62700 1 -- 172.21.15.79:0/3453157710 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689415cfa0 con 0x7f68940cd900 2023-06-23T18:36:43.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f6899363700 1 --2- 172.21.15.79:0/3453157710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6894159170 0x7f68940cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:43.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f6898b62700 1 --2- 172.21.15.79:0/3453157710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68940cd900 0x7f68940ce0e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f689000bbc0 tx=0x7f689000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78a14f9b6a8c7b9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:43.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f688bfff700 1 -- 172.21.15.79:0/3453157710 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689000dd50 con 0x7f68940cd900 2023-06-23T18:36:43.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f688bfff700 1 -- 172.21.15.79:0/3453157710 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6890010470 con 0x7f68940cd900 2023-06-23T18:36:43.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f688bfff700 1 -- 172.21.15.79:0/3453157710 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890010800 con 0x7f68940cd900 2023-06-23T18:36:43.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f689b5c7700 1 -- 172.21.15.79:0/3453157710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68940cd900 msgr2=0x7f68940ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f689b5c7700 1 --2- 172.21.15.79:0/3453157710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68940cd900 0x7f68940ce0e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f689000bbc0 tx=0x7f689000bed0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f689b5c7700 1 -- 172.21.15.79:0/3453157710 shutdown_connections 2023-06-23T18:36:43.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f689b5c7700 1 --2- 172.21.15.79:0/3453157710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6894159170 0x7f68940cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f689b5c7700 1 --2- 172.21.15.79:0/3453157710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68940cd900 0x7f68940ce0e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.342+0000 7f689b5c7700 1 -- 172.21.15.79:0/3453157710 >> 172.21.15.79:0/3453157710 conn(0x7f689406d660 msgr2=0x7f689406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:43.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.343+0000 7f689b5c7700 1 -- 172.21.15.79:0/3453157710 shutdown_connections 2023-06-23T18:36:43.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.343+0000 7f689b5c7700 1 -- 172.21.15.79:0/3453157710 wait complete. 2023-06-23T18:36:43.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.344+0000 7f689b5c7700 1 Processor -- start 2023-06-23T18:36:43.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.344+0000 7f689b5c7700 1 -- start start 2023-06-23T18:36:43.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.344+0000 7f689b5c7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68940cd900 0x7f68941577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.344+0000 7f689b5c7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68940ce620 0x7f68940b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.344+0000 7f689b5c7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6894159170 0x7f68940b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.344+0000 7f689b5c7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6894140ae0 con 0x7f6894159170 2023-06-23T18:36:43.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.344+0000 7f689b5c7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6894157cf0 con 0x7f68940cd900 2023-06-23T18:36:43.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.344+0000 7f689b5c7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6894157f90 con 0x7f68940ce620 2023-06-23T18:36:43.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.344+0000 7f6899b64700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6894159170 0x7f68940b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f6899b64700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6894159170 0x7f68940b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41008/0 (socket says 172.21.15.79:41008) 2023-06-23T18:36:43.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f6899b64700 1 -- 172.21.15.79:0/3743395936 learned_addr learned my addr 172.21.15.79:0/3743395936 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:43.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f6898b62700 1 --2- 172.21.15.79:0/3743395936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68940ce620 0x7f68940b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f6899363700 1 --2- 172.21.15.79:0/3743395936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68940cd900 0x7f68941577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f6899b64700 1 -- 172.21.15.79:0/3743395936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68940ce620 msgr2=0x7f68940b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f6899b64700 1 --2- 172.21.15.79:0/3743395936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68940ce620 0x7f68940b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f6899b64700 1 -- 172.21.15.79:0/3743395936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68940cd900 msgr2=0x7f68941577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f6899b64700 1 --2- 172.21.15.79:0/3743395936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68940cd900 0x7f68941577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f6899b64700 1 -- 172.21.15.79:0/3743395936 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689000b7f0 con 0x7f6894159170 2023-06-23T18:36:43.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f6899b64700 1 --2- 172.21.15.79:0/3743395936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6894159170 0x7f68940b2b60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f688c00cc50 tx=0x7f688c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:43.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f6889ffb700 1 -- 172.21.15.79:0/3743395936 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c003c00 con 0x7f6894159170 2023-06-23T18:36:43.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f6889ffb700 1 -- 172.21.15.79:0/3743395936 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688c003d60 con 0x7f6894159170 2023-06-23T18:36:43.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f689b5c7700 1 -- 172.21.15.79:0/3743395936 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6894158270 con 0x7f6894159170 2023-06-23T18:36:43.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f6889ffb700 1 -- 172.21.15.79:0/3743395936 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c00aab0 con 0x7f6894159170 2023-06-23T18:36:43.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.345+0000 7f689b5c7700 1 -- 172.21.15.79:0/3743395936 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6894158830 con 0x7f6894159170 2023-06-23T18:36:43.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.346+0000 7f689b5c7700 1 -- 172.21.15.79:0/3743395936 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689404e240 con 0x7f6894159170 2023-06-23T18:36:43.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.347+0000 7f6889ffb700 1 -- 172.21.15.79:0/3743395936 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f688c00ac10 con 0x7f6894159170 2023-06-23T18:36:43.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.347+0000 7f6889ffb700 1 --2- 172.21.15.79:0/3743395936 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f687806ace0 0x7f687806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.347+0000 7f6889ffb700 1 -- 172.21.15.79:0/3743395936 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f688c089450 con 0x7f6894159170 2023-06-23T18:36:43.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.347+0000 7f6899363700 1 --2- 172.21.15.79:0/3743395936 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f687806ace0 0x7f687806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.347+0000 7f6899363700 1 --2- 172.21.15.79:0/3743395936 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f687806ace0 0x7f687806d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6884006f10 tx=0x7f6884008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:43.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.351+0000 7f6889ffb700 1 -- 172.21.15.79:0/3743395936 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f688c0544c0 con 0x7f6894159170 2023-06-23T18:36:43.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.555+0000 7f689b5c7700 1 -- 172.21.15.79:0/3743395936 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6894140c20 con 0x7f6894159170 2023-06-23T18:36:43.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.555+0000 7f6889ffb700 1 -- 172.21.15.79:0/3743395936 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f688c057ae0 con 0x7f6894159170 2023-06-23T18:36:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.557+0000 7f689b5c7700 1 -- 172.21.15.79:0/3743395936 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f687806ace0 msgr2=0x7f687806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.557+0000 7f689b5c7700 1 --2- 172.21.15.79:0/3743395936 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f687806ace0 0x7f687806d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6884006f10 tx=0x7f6884008040 comp rx=0 tx=0).stop 2023-06-23T18:36:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.557+0000 7f689b5c7700 1 -- 172.21.15.79:0/3743395936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6894159170 msgr2=0x7f68940b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.557+0000 7f689b5c7700 1 --2- 172.21.15.79:0/3743395936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6894159170 0x7f68940b2b60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f688c00cc50 tx=0x7f688c00cf60 comp rx=0 tx=0).stop 2023-06-23T18:36:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.557+0000 7f689b5c7700 1 -- 172.21.15.79:0/3743395936 shutdown_connections 2023-06-23T18:36:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.557+0000 7f689b5c7700 1 --2- 172.21.15.79:0/3743395936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6894159170 0x7f68940b2b60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.557+0000 7f689b5c7700 1 --2- 172.21.15.79:0/3743395936 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f687806ace0 0x7f687806d190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.557+0000 7f689b5c7700 1 --2- 172.21.15.79:0/3743395936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68940cd900 0x7f68941577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.557+0000 7f689b5c7700 1 --2- 172.21.15.79:0/3743395936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68940ce620 0x7f68940b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.557+0000 7f689b5c7700 1 -- 172.21.15.79:0/3743395936 >> 172.21.15.79:0/3743395936 conn(0x7f689406d660 msgr2=0x7f68940cb650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.558+0000 7f689b5c7700 1 -- 172.21.15.79:0/3743395936 shutdown_connections 2023-06-23T18:36:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.558+0000 7f689b5c7700 1 -- 172.21.15.79:0/3743395936 wait complete. 2023-06-23T18:36:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-23T18:36:43.574 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-06-23T18:36:43.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:43.575 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-06-23T18:36:43.619 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:36:43.619 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:36:43.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:36:43.620 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-06-23T18:36:43.678 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-06-23T18:36:43.679 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:36:43.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.794+0000 7f842cbaf700 1 Processor -- start 2023-06-23T18:36:43.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.794+0000 7f842cbaf700 1 -- start start 2023-06-23T18:36:43.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.795+0000 7f842cbaf700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84280b1270 0x7f842815c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.795+0000 7f842cbaf700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f84280aee30 0x7f842815cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.795+0000 7f842cbaf700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84281608f0 con 0x7f8428071490 2023-06-23T18:36:43.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.795+0000 7f842cbaf700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8428160e20 con 0x7f84280aee30 2023-06-23T18:36:43.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.795+0000 7f842cbaf700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84280cd4f0 con 0x7f84280b1270 2023-06-23T18:36:43.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.795+0000 7f8426d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f84280aee30 0x7f842815cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.796+0000 7f8426d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f84280aee30 0x7f842815cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41016/0 (socket says 172.21.15.79:41016) 2023-06-23T18:36:43.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.795+0000 7f842659c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f8428071490 0x7f8428071860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44054/0 (socket says 172.21.15.79:44054) 2023-06-23T18:36:43.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.796+0000 7f8426d9d700 1 -- 172.21.15.79:0/3275284050 learned_addr learned my addr 172.21.15.79:0/3275284050 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:43.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.796+0000 7f8425d9b700 1 --2- 172.21.15.79:0/3275284050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84280b1270 0x7f842815c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.796+0000 7f8426d9d700 1 -- 172.21.15.79:0/3275284050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84280b1270 msgr2=0x7f842815c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:43.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.796+0000 7f8426d9d700 1 --2- 172.21.15.79:0/3275284050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84280b1270 0x7f842815c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.796+0000 7f8426d9d700 1 -- 172.21.15.79:0/3275284050 >> v1:172.21.15.79:6789/0 conn(0x7f8428071490 legacy=0x7f8428071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.796+0000 7f8426d9d700 1 -- 172.21.15.79:0/3275284050 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8428160b40 con 0x7f84280aee30 2023-06-23T18:36:43.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.796+0000 7f842559a700 1 -- 172.21.15.79:0/3275284050 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066187846 0 0) 0x7f84281608f0 con 0x7f8428071490 2023-06-23T18:36:43.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.796+0000 7f8425d9b700 1 --2- 172.21.15.79:0/3275284050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84280b1270 0x7f842815c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:43.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.796+0000 7f8426d9d700 1 --2- 172.21.15.79:0/3275284050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84280aee30 0x7f842815cf30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8420009e50 tx=0x7f842000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd0f8d379bf580 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:43.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.796+0000 7f842559a700 1 -- 172.21.15.79:0/3275284050 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842000e070 con 0x7f84280aee30 2023-06-23T18:36:43.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.796+0000 7f842559a700 1 -- 172.21.15.79:0/3275284050 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842000b9c0 con 0x7f84280aee30 2023-06-23T18:36:43.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.796+0000 7f842559a700 1 -- 172.21.15.79:0/3275284050 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842000bd70 con 0x7f84280aee30 2023-06-23T18:36:43.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.797+0000 7f842cbaf700 1 -- 172.21.15.79:0/3275284050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84280aee30 msgr2=0x7f842815cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.797+0000 7f842cbaf700 1 --2- 172.21.15.79:0/3275284050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84280aee30 0x7f842815cf30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8420009e50 tx=0x7f842000b330 comp rx=0 tx=0).stop 2023-06-23T18:36:43.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.797+0000 7f842cbaf700 1 -- 172.21.15.79:0/3275284050 shutdown_connections 2023-06-23T18:36:43.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.797+0000 7f842cbaf700 1 --2- 172.21.15.79:0/3275284050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84280aee30 0x7f842815cf30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.797+0000 7f842cbaf700 1 --2- 172.21.15.79:0/3275284050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84280b1270 0x7f842815c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.797+0000 7f842cbaf700 1 -- 172.21.15.79:0/3275284050 >> 172.21.15.79:0/3275284050 conn(0x7f842806d660 msgr2=0x7f842806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:43.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.797+0000 7f842cbaf700 1 -- 172.21.15.79:0/3275284050 shutdown_connections 2023-06-23T18:36:43.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.797+0000 7f842cbaf700 1 -- 172.21.15.79:0/3275284050 wait complete. 2023-06-23T18:36:43.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.798+0000 7f842cbaf700 1 Processor -- start 2023-06-23T18:36:43.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.798+0000 7f842cbaf700 1 -- start start 2023-06-23T18:36:43.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f842cbaf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8428071490 0x7f842814a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f842cbaf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84280aee30 0x7f84280b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f842cbaf700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84280b1270 0x7f84280c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f842cbaf700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842814d3a0 con 0x7f84280aee30 2023-06-23T18:36:43.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f842cbaf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84280c44d0 con 0x7f8428071490 2023-06-23T18:36:43.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f842cbaf700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84280c41f0 con 0x7f84280b1270 2023-06-23T18:36:43.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f8425d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84280aee30 0x7f84280b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f8425d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84280aee30 0x7f84280b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41020/0 (socket says 172.21.15.79:41020) 2023-06-23T18:36:43.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f8425d9b700 1 -- 172.21.15.79:0/1698269083 learned_addr learned my addr 172.21.15.79:0/1698269083 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:43.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f842659c700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8428071490 0x7f842814a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f8426d9d700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84280b1270 0x7f84280c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f8425d9b700 1 -- 172.21.15.79:0/1698269083 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84280b1270 msgr2=0x7f84280c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f8425d9b700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84280b1270 0x7f84280c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f8425d9b700 1 -- 172.21.15.79:0/1698269083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8428071490 msgr2=0x7f842814a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:43.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f8425d9b700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8428071490 0x7f842814a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:43.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f8425d9b700 1 -- 172.21.15.79:0/1698269083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8420009b30 con 0x7f84280aee30 2023-06-23T18:36:43.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f842659c700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8428071490 0x7f842814a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:43.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f8426d9d700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84280b1270 0x7f84280c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:43.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.799+0000 7f8425d9b700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84280aee30 0x7f84280b2620 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f841800e810 tx=0x7f841800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:43.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.800+0000 7f8416ffd700 1 -- 172.21.15.79:0/1698269083 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418013610 con 0x7f84280aee30 2023-06-23T18:36:43.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.800+0000 7f8416ffd700 1 -- 172.21.15.79:0/1698269083 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8418013770 con 0x7f84280aee30 2023-06-23T18:36:43.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.800+0000 7f842cbaf700 1 -- 172.21.15.79:0/1698269083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84280c3450 con 0x7f84280aee30 2023-06-23T18:36:43.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.800+0000 7f8416ffd700 1 -- 172.21.15.79:0/1698269083 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418013b20 con 0x7f84280aee30 2023-06-23T18:36:43.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.800+0000 7f842cbaf700 1 -- 172.21.15.79:0/1698269083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84280c3a10 con 0x7f84280aee30 2023-06-23T18:36:43.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.801+0000 7f842cbaf700 1 -- 172.21.15.79:0/1698269083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842804e240 con 0x7f84280aee30 2023-06-23T18:36:43.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.801+0000 7f8416ffd700 1 -- 172.21.15.79:0/1698269083 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8418013c80 con 0x7f84280aee30 2023-06-23T18:36:43.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.801+0000 7f8416ffd700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f840806ace0 0x7f840806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:43.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.801+0000 7f8416ffd700 1 -- 172.21.15.79:0/1698269083 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8418089150 con 0x7f84280aee30 2023-06-23T18:36:43.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.802+0000 7f842659c700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f840806ace0 0x7f840806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:43.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.802+0000 7f842659c700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f840806ace0 0x7f840806d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f841c000f60 tx=0x7f841c007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:43.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:43.806+0000 7f8416ffd700 1 -- 172.21.15.79:0/1698269083 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8418054250 con 0x7f84280aee30 2023-06-23T18:36:44.038 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.034+0000 7f842cbaf700 1 -- 172.21.15.79:0/1698269083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84280724a0 con 0x7f84280aee30 2023-06-23T18:36:44.039 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.034+0000 7f8416ffd700 1 -- 172.21.15.79:0/1698269083 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f8418057870 con 0x7f84280aee30 2023-06-23T18:36:44.040 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:44.040 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/399719238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":399719238},{"type":"v1","addr":"172.21.15.155:6834","nonce":399719238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1846440916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1846440916},{"type":"v1","addr":"172.21.15.155:6835","nonce":1846440916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3310180303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3310180303},{"type":"v1","addr":"172.21.15.79:6835","nonce":3310180303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:35:53.511495+0000","modified":"2023-06-23T18:35:56.527609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6837/3900137670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3900137670},{"type":"v1","addr":"172.21.15.79:6837","nonce":3900137670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:36:44.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.036+0000 7f842cbaf700 1 -- 172.21.15.79:0/1698269083 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f840806ace0 msgr2=0x7f840806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.036+0000 7f842cbaf700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f840806ace0 0x7f840806d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f841c000f60 tx=0x7f841c007d30 comp rx=0 tx=0).stop 2023-06-23T18:36:44.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.036+0000 7f842cbaf700 1 -- 172.21.15.79:0/1698269083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84280aee30 msgr2=0x7f84280b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.036+0000 7f842cbaf700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84280aee30 0x7f84280b2620 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f841800e810 tx=0x7f841800ebd0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.036+0000 7f842cbaf700 1 -- 172.21.15.79:0/1698269083 shutdown_connections 2023-06-23T18:36:44.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.036+0000 7f842cbaf700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84280b1270 0x7f84280c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.036+0000 7f842cbaf700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f840806ace0 0x7f840806d190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.036+0000 7f842cbaf700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8428071490 0x7f842814a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.036+0000 7f842cbaf700 1 --2- 172.21.15.79:0/1698269083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84280aee30 0x7f84280b2620 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.036+0000 7f842cbaf700 1 -- 172.21.15.79:0/1698269083 >> 172.21.15.79:0/1698269083 conn(0x7f842806d660 msgr2=0x7f84280b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:44.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.036+0000 7f842cbaf700 1 -- 172.21.15.79:0/1698269083 shutdown_connections 2023-06-23T18:36:44.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.036+0000 7f842cbaf700 1 -- 172.21.15.79:0/1698269083 wait complete. 2023-06-23T18:36:44.045 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 8 2023-06-23T18:36:44.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:44.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.169+0000 7fdf6a61e700 1 Processor -- start 2023-06-23T18:36:44.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.169+0000 7fdf6a61e700 1 -- start start 2023-06-23T18:36:44.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf6a61e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdf640cec00 0x7fdf640d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf6a61e700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdf640ae880 0x7fdf640b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf6a61e700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf6415a620 con 0x7fdf640af410 2023-06-23T18:36:44.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf6a61e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf640b1770 con 0x7fdf640cec00 2023-06-23T18:36:44.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf6a61e700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf640b1a50 con 0x7fdf640ae880 2023-06-23T18:36:44.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf637fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdf640cec00 0x7fdf640d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf637fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdf640cec00 0x7fdf640d0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41028/0 (socket says 172.21.15.79:41028) 2023-06-23T18:36:44.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf637fe700 1 -- 172.21.15.79:0/270456658 learned_addr learned my addr 172.21.15.79:0/270456658 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:44.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf637fe700 1 -- 172.21.15.79:0/270456658 >> v2:172.21.15.155:3300/0 conn(0x7fdf640ae880 msgr2=0x7fdf640b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:44.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf637fe700 1 --2- 172.21.15.79:0/270456658 >> v2:172.21.15.155:3300/0 conn(0x7fdf640ae880 0x7fdf640b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf637fe700 1 -- 172.21.15.79:0/270456658 >> v1:172.21.15.155:6789/0 conn(0x7fdf640af410 legacy=0x7fdf640ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:44.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf637fe700 1 -- 172.21.15.79:0/270456658 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640b1d30 con 0x7fdf640cec00 2023-06-23T18:36:44.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf637fe700 1 --2- 172.21.15.79:0/270456658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf640cec00 0x7fdf640d0ff0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdf6406e160 tx=0x7fdf54009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada3798cd87685d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:44.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf62ffd700 1 -- 172.21.15.79:0/270456658 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54004230 con 0x7fdf640cec00 2023-06-23T18:36:44.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf62ffd700 1 -- 172.21.15.79:0/270456658 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf54004390 con 0x7fdf640cec00 2023-06-23T18:36:44.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.170+0000 7fdf62ffd700 1 -- 172.21.15.79:0/270456658 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf540036a0 con 0x7fdf640cec00 2023-06-23T18:36:44.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.171+0000 7fdf6a61e700 1 -- 172.21.15.79:0/270456658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf640cec00 msgr2=0x7fdf640d0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.171+0000 7fdf6a61e700 1 --2- 172.21.15.79:0/270456658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf640cec00 0x7fdf640d0ff0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdf6406e160 tx=0x7fdf54009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.171+0000 7fdf6a61e700 1 -- 172.21.15.79:0/270456658 shutdown_connections 2023-06-23T18:36:44.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.171+0000 7fdf6a61e700 1 --2- 172.21.15.79:0/270456658 >> v2:172.21.15.155:3300/0 conn(0x7fdf640ae880 0x7fdf640b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.171+0000 7fdf6a61e700 1 --2- 172.21.15.79:0/270456658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf640cec00 0x7fdf640d0ff0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.171+0000 7fdf6a61e700 1 -- 172.21.15.79:0/270456658 >> 172.21.15.79:0/270456658 conn(0x7fdf64089fc0 msgr2=0x7fdf64099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:44.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.171+0000 7fdf6a61e700 1 -- 172.21.15.79:0/270456658 shutdown_connections 2023-06-23T18:36:44.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.171+0000 7fdf6a61e700 1 -- 172.21.15.79:0/270456658 wait complete. 2023-06-23T18:36:44.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.172+0000 7fdf6a61e700 1 Processor -- start 2023-06-23T18:36:44.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.172+0000 7fdf6a61e700 1 -- start start 2023-06-23T18:36:44.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.173+0000 7fdf6a61e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf640ae880 0x7fdf64159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.173+0000 7fdf6a61e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf640af410 0x7fdf6406c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.173+0000 7fdf6a61e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf640cec00 0x7fdf640c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.173+0000 7fdf6a61e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf64158f70 con 0x7fdf640ae880 2023-06-23T18:36:44.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.173+0000 7fdf6a61e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6406cfc0 con 0x7fdf640af410 2023-06-23T18:36:44.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.173+0000 7fdf6a61e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640cd6e0 con 0x7fdf640cec00 2023-06-23T18:36:44.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.173+0000 7fdf63fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf640ae880 0x7fdf64159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.173+0000 7fdf63fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf640ae880 0x7fdf64159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41032/0 (socket says 172.21.15.79:41032) 2023-06-23T18:36:44.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.173+0000 7fdf63fff700 1 -- 172.21.15.79:0/3499125750 learned_addr learned my addr 172.21.15.79:0/3499125750 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:44.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.173+0000 7fdf637fe700 1 --2- 172.21.15.79:0/3499125750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf640af410 0x7fdf6406c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.173+0000 7fdf68bbb700 1 --2- 172.21.15.79:0/3499125750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf640cec00 0x7fdf640c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.174+0000 7fdf63fff700 1 -- 172.21.15.79:0/3499125750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf640cec00 msgr2=0x7fdf640c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.174+0000 7fdf63fff700 1 --2- 172.21.15.79:0/3499125750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf640cec00 0x7fdf640c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.174+0000 7fdf63fff700 1 -- 172.21.15.79:0/3499125750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf640af410 msgr2=0x7fdf6406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.174+0000 7fdf63fff700 1 --2- 172.21.15.79:0/3499125750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf640af410 0x7fdf6406c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.174+0000 7fdf63fff700 1 -- 172.21.15.79:0/3499125750 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf54009550 con 0x7fdf640ae880 2023-06-23T18:36:44.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.174+0000 7fdf63fff700 1 --2- 172.21.15.79:0/3499125750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf640ae880 0x7fdf64159110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c520 tx=0x7fdf4c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:44.191 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.174+0000 7fdf60ff9700 1 -- 172.21.15.79:0/3499125750 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0054e0 con 0x7fdf640ae880 2023-06-23T18:36:44.191 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.174+0000 7fdf60ff9700 1 -- 172.21.15.79:0/3499125750 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c0036d0 con 0x7fdf640ae880 2023-06-23T18:36:44.191 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.174+0000 7fdf6a61e700 1 -- 172.21.15.79:0/3499125750 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf640cc940 con 0x7fdf640ae880 2023-06-23T18:36:44.191 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.174+0000 7fdf60ff9700 1 -- 172.21.15.79:0/3499125750 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c003a80 con 0x7fdf640ae880 2023-06-23T18:36:44.191 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.174+0000 7fdf6a61e700 1 -- 172.21.15.79:0/3499125750 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf640ccf00 con 0x7fdf640ae880 2023-06-23T18:36:44.191 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.175+0000 7fdf6a61e700 1 -- 172.21.15.79:0/3499125750 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf640cd550 con 0x7fdf640ae880 2023-06-23T18:36:44.191 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.175+0000 7fdf60ff9700 1 -- 172.21.15.79:0/3499125750 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdf4c003be0 con 0x7fdf640ae880 2023-06-23T18:36:44.191 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.176+0000 7fdf60ff9700 1 --2- 172.21.15.79:0/3499125750 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdf4406ace0 0x7fdf4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.176+0000 7fdf60ff9700 1 -- 172.21.15.79:0/3499125750 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fdf4c088930 con 0x7fdf640ae880 2023-06-23T18:36:44.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.176+0000 7fdf637fe700 1 --2- 172.21.15.79:0/3499125750 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdf4406ace0 0x7fdf4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.176+0000 7fdf637fe700 1 --2- 172.21.15.79:0/3499125750 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdf4406ace0 0x7fdf4406d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009130 tx=0x7fdf54008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:44.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.180+0000 7fdf60ff9700 1 -- 172.21.15.79:0/3499125750 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdf4c053900 con 0x7fdf640ae880 2023-06-23T18:36:44.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.369+0000 7fdf6a61e700 1 -- 172.21.15.79:0/3499125750 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf64065ea0 con 0x7fdf640ae880 2023-06-23T18:36:44.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.370+0000 7fdf60ff9700 1 -- 172.21.15.79:0/3499125750 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18329 (secure 0 0 0) 0x7fdf4c017090 con 0x7fdf640ae880 2023-06-23T18:36:44.375 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:44.376 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":22,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:36:42.066263+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1a","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:44.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.372+0000 7fdf6a61e700 1 -- 172.21.15.79:0/3499125750 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdf4406ace0 msgr2=0x7fdf4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.372+0000 7fdf6a61e700 1 --2- 172.21.15.79:0/3499125750 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdf4406ace0 0x7fdf4406d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009130 tx=0x7fdf54008030 comp rx=0 tx=0).stop 2023-06-23T18:36:44.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.372+0000 7fdf6a61e700 1 -- 172.21.15.79:0/3499125750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf640ae880 msgr2=0x7fdf64159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.372+0000 7fdf6a61e700 1 --2- 172.21.15.79:0/3499125750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf640ae880 0x7fdf64159110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c520 tx=0x7fdf4c00c830 comp rx=0 tx=0).stop 2023-06-23T18:36:44.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.372+0000 7fdf6a61e700 1 -- 172.21.15.79:0/3499125750 shutdown_connections 2023-06-23T18:36:44.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.372+0000 7fdf6a61e700 1 --2- 172.21.15.79:0/3499125750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf640cec00 0x7fdf640c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.372+0000 7fdf6a61e700 1 --2- 172.21.15.79:0/3499125750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf640ae880 0x7fdf64159110 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.372+0000 7fdf6a61e700 1 --2- 172.21.15.79:0/3499125750 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdf4406ace0 0x7fdf4406d190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.372+0000 7fdf6a61e700 1 --2- 172.21.15.79:0/3499125750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf640af410 0x7fdf6406c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.372+0000 7fdf6a61e700 1 -- 172.21.15.79:0/3499125750 >> 172.21.15.79:0/3499125750 conn(0x7fdf64089fc0 msgr2=0x7fdf640cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:44.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.373+0000 7fdf6a61e700 1 -- 172.21.15.79:0/3499125750 shutdown_connections 2023-06-23T18:36:44.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.373+0000 7fdf6a61e700 1 -- 172.21.15.79:0/3499125750 wait complete. 2023-06-23T18:36:44.390 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:35:51.125894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:35:52.478806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:44.391 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:44.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.508+0000 7f7f4253b700 1 Processor -- start 2023-06-23T18:36:44.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.508+0000 7f7f4253b700 1 -- start start 2023-06-23T18:36:44.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.509+0000 7f7f4253b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7f3c157c40 0x7f7f3c0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.509+0000 7f7f4253b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c071500 0x7f7f3c15dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.509+0000 7f7f4253b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3c15e5d0 con 0x7f7f3c15e170 2023-06-23T18:36:44.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.509+0000 7f7f4253b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f3c0b1400 con 0x7f7f3c157c40 2023-06-23T18:36:44.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.509+0000 7f7f4253b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c0c9c40 con 0x7f7f3c071500 2023-06-23T18:36:44.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.510+0000 7f7f3bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7f3c157c40 0x7f7f3c0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.510+0000 7f7f3bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7f3c157c40 0x7f7f3c0c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41040/0 (socket says 172.21.15.79:41040) 2023-06-23T18:36:44.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.510+0000 7f7f3bfff700 1 -- 172.21.15.79:0/1866429579 learned_addr learned my addr 172.21.15.79:0/1866429579 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:44.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.510+0000 7f7f3b7fe700 1 --2- 172.21.15.79:0/1866429579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c071500 0x7f7f3c15dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.510+0000 7f7f3bfff700 1 -- 172.21.15.79:0/1866429579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c071500 msgr2=0x7f7f3c15dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:44.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.510+0000 7f7f3bfff700 1 --2- 172.21.15.79:0/1866429579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c071500 0x7f7f3c15dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.510+0000 7f7f3bfff700 1 -- 172.21.15.79:0/1866429579 >> v1:172.21.15.155:6789/0 conn(0x7f7f3c15e170 legacy=0x7f7f3c15a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:44.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.510+0000 7f7f3bfff700 1 -- 172.21.15.79:0/1866429579 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c158010 con 0x7f7f3c157c40 2023-06-23T18:36:44.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.510+0000 7f7f3bfff700 1 --2- 172.21.15.79:0/1866429579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f3c157c40 0x7f7f3c0c9620 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0098f0 tx=0x7f7f2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56f93b40ad89554 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:44.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.510+0000 7f7f3affd700 1 -- 172.21.15.79:0/1866429579 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c004230 con 0x7f7f3c157c40 2023-06-23T18:36:44.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.510+0000 7f7f3affd700 1 -- 172.21.15.79:0/1866429579 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c004390 con 0x7f7f3c157c40 2023-06-23T18:36:44.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.510+0000 7f7f3affd700 1 -- 172.21.15.79:0/1866429579 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0036a0 con 0x7f7f3c157c40 2023-06-23T18:36:44.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.511+0000 7f7f4253b700 1 -- 172.21.15.79:0/1866429579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f3c157c40 msgr2=0x7f7f3c0c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.511+0000 7f7f4253b700 1 --2- 172.21.15.79:0/1866429579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f3c157c40 0x7f7f3c0c9620 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0098f0 tx=0x7f7f2c009c00 comp rx=0 tx=0).stop 2023-06-23T18:36:44.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.511+0000 7f7f4253b700 1 -- 172.21.15.79:0/1866429579 shutdown_connections 2023-06-23T18:36:44.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.511+0000 7f7f4253b700 1 --2- 172.21.15.79:0/1866429579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f3c157c40 0x7f7f3c0c9620 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.511+0000 7f7f4253b700 1 --2- 172.21.15.79:0/1866429579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c071500 0x7f7f3c15dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.511+0000 7f7f4253b700 1 -- 172.21.15.79:0/1866429579 >> 172.21.15.79:0/1866429579 conn(0x7f7f3c0cfc60 msgr2=0x7f7f3c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:44.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.511+0000 7f7f4253b700 1 -- 172.21.15.79:0/1866429579 shutdown_connections 2023-06-23T18:36:44.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.511+0000 7f7f4253b700 1 -- 172.21.15.79:0/1866429579 wait complete. 2023-06-23T18:36:44.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.512+0000 7f7f4253b700 1 Processor -- start 2023-06-23T18:36:44.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.512+0000 7f7f4253b700 1 -- start start 2023-06-23T18:36:44.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f4253b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c071500 0x7f7f3c152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f4253b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c157c40 0x7f7f3c1e5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f4253b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f3c15e170 0x7f7f3c1566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f4253b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c1537c0 con 0x7f7f3c15e170 2023-06-23T18:36:44.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f4253b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c0c2a30 con 0x7f7f3c157c40 2023-06-23T18:36:44.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f4253b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c0c2d10 con 0x7f7f3c071500 2023-06-23T18:36:44.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f40ad8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f3c15e170 0x7f7f3c1566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f40ad8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f3c15e170 0x7f7f3c1566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41050/0 (socket says 172.21.15.79:41050) 2023-06-23T18:36:44.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f40ad8700 1 -- 172.21.15.79:0/3948324783 learned_addr learned my addr 172.21.15.79:0/3948324783 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:44.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f3bfff700 1 --2- 172.21.15.79:0/3948324783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c071500 0x7f7f3c152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f3b7fe700 1 --2- 172.21.15.79:0/3948324783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c157c40 0x7f7f3c1e5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f40ad8700 1 -- 172.21.15.79:0/3948324783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c071500 msgr2=0x7f7f3c152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f40ad8700 1 --2- 172.21.15.79:0/3948324783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c071500 0x7f7f3c152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f40ad8700 1 -- 172.21.15.79:0/3948324783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c157c40 msgr2=0x7f7f3c1e5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f40ad8700 1 --2- 172.21.15.79:0/3948324783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c157c40 0x7f7f3c1e5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f40ad8700 1 -- 172.21.15.79:0/3948324783 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c009550 con 0x7f7f3c15e170 2023-06-23T18:36:44.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.513+0000 7f7f40ad8700 1 --2- 172.21.15.79:0/3948324783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f3c15e170 0x7f7f3c1566a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400b970 tx=0x7f7f3400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:44.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.514+0000 7f7f38ff9700 1 -- 172.21.15.79:0/3948324783 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3400c9b0 con 0x7f7f3c15e170 2023-06-23T18:36:44.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.514+0000 7f7f38ff9700 1 -- 172.21.15.79:0/3948324783 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3400cb10 con 0x7f7f3c15e170 2023-06-23T18:36:44.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.514+0000 7f7f38ff9700 1 -- 172.21.15.79:0/3948324783 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f34014440 con 0x7f7f3c15e170 2023-06-23T18:36:44.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.514+0000 7f7f4253b700 1 -- 172.21.15.79:0/3948324783 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3c0c2ff0 con 0x7f7f3c15e170 2023-06-23T18:36:44.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.514+0000 7f7f4253b700 1 -- 172.21.15.79:0/3948324783 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3c1e5740 con 0x7f7f3c15e170 2023-06-23T18:36:44.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.515+0000 7f7f4253b700 1 -- 172.21.15.79:0/3948324783 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f3c0672c0 con 0x7f7f3c15e170 2023-06-23T18:36:44.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.515+0000 7f7f38ff9700 1 -- 172.21.15.79:0/3948324783 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f3401d3b0 con 0x7f7f3c15e170 2023-06-23T18:36:44.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.515+0000 7f7f38ff9700 1 --2- 172.21.15.79:0/3948324783 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7f2806ace0 0x7f7f2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.516+0000 7f7f38ff9700 1 -- 172.21.15.79:0/3948324783 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7f340890e0 con 0x7f7f3c15e170 2023-06-23T18:36:44.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.516+0000 7f7f3bfff700 1 --2- 172.21.15.79:0/3948324783 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7f2806ace0 0x7f7f2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.516+0000 7f7f3bfff700 1 --2- 172.21.15.79:0/3948324783 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7f2806ace0 0x7f7f2806d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009130 tx=0x7f7f2c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:44.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.520+0000 7f7f38ff9700 1 -- 172.21.15.79:0/3948324783 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7f340541e0 con 0x7f7f3c15e170 2023-06-23T18:36:44.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.705+0000 7f7f4253b700 1 -- 172.21.15.79:0/3948324783 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7f3c0cbc30 con 0x7f7f3c15e170 2023-06-23T18:36:44.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.706+0000 7f7f38ff9700 1 -- 172.21.15.79:0/3948324783 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7f34057800 con 0x7f7f3c15e170 2023-06-23T18:36:44.711 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:44.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.708+0000 7f7f4253b700 1 -- 172.21.15.79:0/3948324783 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7f2806ace0 msgr2=0x7f7f2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.708+0000 7f7f4253b700 1 --2- 172.21.15.79:0/3948324783 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7f2806ace0 0x7f7f2806d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009130 tx=0x7f7f2c008030 comp rx=0 tx=0).stop 2023-06-23T18:36:44.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.708+0000 7f7f4253b700 1 -- 172.21.15.79:0/3948324783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f3c15e170 msgr2=0x7f7f3c1566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.708+0000 7f7f4253b700 1 --2- 172.21.15.79:0/3948324783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f3c15e170 0x7f7f3c1566a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400b970 tx=0x7f7f3400bd30 comp rx=0 tx=0).stop 2023-06-23T18:36:44.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.708+0000 7f7f4253b700 1 -- 172.21.15.79:0/3948324783 shutdown_connections 2023-06-23T18:36:44.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.708+0000 7f7f4253b700 1 --2- 172.21.15.79:0/3948324783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f3c15e170 0x7f7f3c1566a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.708+0000 7f7f4253b700 1 --2- 172.21.15.79:0/3948324783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f3c071500 0x7f7f3c152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.708+0000 7f7f4253b700 1 --2- 172.21.15.79:0/3948324783 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7f2806ace0 0x7f7f2806d190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.708+0000 7f7f4253b700 1 --2- 172.21.15.79:0/3948324783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f3c157c40 0x7f7f3c1e5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.708+0000 7f7f4253b700 1 -- 172.21.15.79:0/3948324783 >> 172.21.15.79:0/3948324783 conn(0x7f7f3c0cfc60 msgr2=0x7f7f3c0ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:44.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.708+0000 7f7f4253b700 1 -- 172.21.15.79:0/3948324783 shutdown_connections 2023-06-23T18:36:44.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.708+0000 7f7f4253b700 1 -- 172.21.15.79:0/3948324783 wait complete. 2023-06-23T18:36:44.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:44.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.843+0000 7f2507392700 1 Processor -- start 2023-06-23T18:36:44.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.844+0000 7f2507392700 1 -- start start 2023-06-23T18:36:44.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.844+0000 7f2507392700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2500156ae0 0x7f250015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.844+0000 7f2507392700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f25000cc4d0 0x7f2500071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.844+0000 7f2507392700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f250015daf0 con 0x7f25000cc060 2023-06-23T18:36:44.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.844+0000 7f2507392700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250015a560 con 0x7f25000cc4d0 2023-06-23T18:36:44.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.844+0000 7f2507392700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2500159950 con 0x7f2500156ae0 2023-06-23T18:36:44.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.845+0000 7f250592f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f25000cc4d0 0x7f2500071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.845+0000 7f250592f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f25000cc4d0 0x7f2500071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41058/0 (socket says 172.21.15.79:41058) 2023-06-23T18:36:44.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.845+0000 7f250592f700 1 -- 172.21.15.79:0/3731453355 learned_addr learned my addr 172.21.15.79:0/3731453355 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:44.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.845+0000 7f250512e700 1 --2- 172.21.15.79:0/3731453355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2500156ae0 0x7f250015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.845+0000 7f250592f700 1 -- 172.21.15.79:0/3731453355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2500156ae0 msgr2=0x7f250015d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:44.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.845+0000 7f250592f700 1 --2- 172.21.15.79:0/3731453355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2500156ae0 0x7f250015d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.845+0000 7f250592f700 1 -- 172.21.15.79:0/3731453355 >> v1:172.21.15.155:6789/0 conn(0x7f25000cc060 legacy=0x7f2500071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.845+0000 7f250592f700 1 -- 172.21.15.79:0/3731453355 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2500159c30 con 0x7f25000cc4d0 2023-06-23T18:36:44.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.845+0000 7f250592f700 1 --2- 172.21.15.79:0/3731453355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25000cc4d0 0x7f2500071e20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009e50 tx=0x7f24f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb55fce12783185 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:44.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.845+0000 7f24f7fff700 1 -- 172.21.15.79:0/3731453355 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f800e070 con 0x7f25000cc4d0 2023-06-23T18:36:44.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.845+0000 7f24f7fff700 1 -- 172.21.15.79:0/3731453355 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f800b9c0 con 0x7f25000cc4d0 2023-06-23T18:36:44.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.845+0000 7f24f7fff700 1 -- 172.21.15.79:0/3731453355 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f800bd70 con 0x7f25000cc4d0 2023-06-23T18:36:44.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.846+0000 7f2507392700 1 -- 172.21.15.79:0/3731453355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25000cc4d0 msgr2=0x7f2500071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.846+0000 7f2507392700 1 --2- 172.21.15.79:0/3731453355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25000cc4d0 0x7f2500071e20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009e50 tx=0x7f24f800b330 comp rx=0 tx=0).stop 2023-06-23T18:36:44.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.846+0000 7f2507392700 1 -- 172.21.15.79:0/3731453355 shutdown_connections 2023-06-23T18:36:44.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.846+0000 7f2507392700 1 --2- 172.21.15.79:0/3731453355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25000cc4d0 0x7f2500071e20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.846+0000 7f2507392700 1 --2- 172.21.15.79:0/3731453355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2500156ae0 0x7f250015d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.846+0000 7f2507392700 1 -- 172.21.15.79:0/3731453355 >> 172.21.15.79:0/3731453355 conn(0x7f250006afa0 msgr2=0x7f250006b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:44.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.846+0000 7f2507392700 1 -- 172.21.15.79:0/3731453355 shutdown_connections 2023-06-23T18:36:44.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.846+0000 7f2507392700 1 -- 172.21.15.79:0/3731453355 wait complete. 2023-06-23T18:36:44.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.847+0000 7f2507392700 1 Processor -- start 2023-06-23T18:36:44.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.847+0000 7f2507392700 1 -- start start 2023-06-23T18:36:44.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f2507392700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25000cc060 0x7f25001ef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f2507392700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25000cc4d0 0x7f25000c3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f2507392700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2500156ae0 0x7f25000c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f2507392700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25000c4950 con 0x7f25000cc060 2023-06-23T18:36:44.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f2507392700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25001ef810 con 0x7f2500156ae0 2023-06-23T18:36:44.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f2507392700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250014b830 con 0x7f25000cc4d0 2023-06-23T18:36:44.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f250512e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25000cc060 0x7f25001ef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f250512e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25000cc060 0x7f25001ef2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41060/0 (socket says 172.21.15.79:41060) 2023-06-23T18:36:44.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f250512e700 1 -- 172.21.15.79:0/2730043230 learned_addr learned my addr 172.21.15.79:0/2730043230 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:44.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f250512e700 1 -- 172.21.15.79:0/2730043230 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25000cc4d0 msgr2=0x7f25000c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f250512e700 1 --2- 172.21.15.79:0/2730043230 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25000cc4d0 0x7f25000c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f250512e700 1 -- 172.21.15.79:0/2730043230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2500156ae0 msgr2=0x7f25000c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:44.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f250512e700 1 --2- 172.21.15.79:0/2730043230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2500156ae0 0x7f25000c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:44.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f250512e700 1 -- 172.21.15.79:0/2730043230 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f8009b30 con 0x7f25000cc060 2023-06-23T18:36:44.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f250512e700 1 --2- 172.21.15.79:0/2730043230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25000cc060 0x7f25001ef2d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f24f000d800 tx=0x7f24f000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:44.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f24f5ffb700 1 -- 172.21.15.79:0/2730043230 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f00125f0 con 0x7f25000cc060 2023-06-23T18:36:44.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f24f5ffb700 1 -- 172.21.15.79:0/2730043230 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f0012750 con 0x7f25000cc060 2023-06-23T18:36:44.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f2507392700 1 -- 172.21.15.79:0/2730043230 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25000c3cd0 con 0x7f25000cc060 2023-06-23T18:36:44.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.848+0000 7f24f5ffb700 1 -- 172.21.15.79:0/2730043230 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0012b00 con 0x7f25000cc060 2023-06-23T18:36:44.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.849+0000 7f2507392700 1 -- 172.21.15.79:0/2730043230 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25000c4290 con 0x7f25000cc060 2023-06-23T18:36:44.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.849+0000 7f2507392700 1 -- 172.21.15.79:0/2730043230 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25000672c0 con 0x7f25000cc060 2023-06-23T18:36:44.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.850+0000 7f24f5ffb700 1 -- 172.21.15.79:0/2730043230 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f24f0012c60 con 0x7f25000cc060 2023-06-23T18:36:44.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.850+0000 7f24f5ffb700 1 --2- 172.21.15.79:0/2730043230 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f24e406ace0 0x7f24e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:44.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.850+0000 7f250492d700 1 --2- 172.21.15.79:0/2730043230 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f24e406ace0 0x7f24e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:44.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.850+0000 7f24f5ffb700 1 -- 172.21.15.79:0/2730043230 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f24f0089820 con 0x7f25000cc060 2023-06-23T18:36:44.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.851+0000 7f250492d700 1 --2- 172.21.15.79:0/2730043230 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f24e406ace0 0x7f24e406d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f24fc000f60 tx=0x7f24fc007a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:44.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:44.855+0000 7f24f5ffb700 1 -- 172.21.15.79:0/2730043230 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f24f0054920 con 0x7f25000cc060 2023-06-23T18:36:45.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.040+0000 7f2507392700 1 -- 172.21.15.79:0/2730043230 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25000b1be0 con 0x7f25000cc060 2023-06-23T18:36:45.046 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.041+0000 7f24f5ffb700 1 -- 172.21.15.79:0/2730043230 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f24f0057f40 con 0x7f25000cc060 2023-06-23T18:36:45.046 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:45.048 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.043+0000 7f2507392700 1 -- 172.21.15.79:0/2730043230 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f24e406ace0 msgr2=0x7f24e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.048 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.043+0000 7f2507392700 1 --2- 172.21.15.79:0/2730043230 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f24e406ace0 0x7f24e406d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f24fc000f60 tx=0x7f24fc007a40 comp rx=0 tx=0).stop 2023-06-23T18:36:45.048 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.043+0000 7f2507392700 1 -- 172.21.15.79:0/2730043230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25000cc060 msgr2=0x7f25001ef2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.048 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.043+0000 7f2507392700 1 --2- 172.21.15.79:0/2730043230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25000cc060 0x7f25001ef2d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f24f000d800 tx=0x7f24f000dbc0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.048 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.043+0000 7f2507392700 1 -- 172.21.15.79:0/2730043230 shutdown_connections 2023-06-23T18:36:45.048 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.043+0000 7f2507392700 1 --2- 172.21.15.79:0/2730043230 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f24e406ace0 0x7f24e406d190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.049 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.043+0000 7f2507392700 1 --2- 172.21.15.79:0/2730043230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2500156ae0 0x7f25000c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.049 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.043+0000 7f2507392700 1 --2- 172.21.15.79:0/2730043230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25000cc060 0x7f25001ef2d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.049 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.043+0000 7f2507392700 1 --2- 172.21.15.79:0/2730043230 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25000cc4d0 0x7f25000c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.049 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.043+0000 7f2507392700 1 -- 172.21.15.79:0/2730043230 >> 172.21.15.79:0/2730043230 conn(0x7f250006afa0 msgr2=0x7f25000cf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:45.049 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.043+0000 7f2507392700 1 -- 172.21.15.79:0/2730043230 shutdown_connections 2023-06-23T18:36:45.049 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.043+0000 7f2507392700 1 -- 172.21.15.79:0/2730043230 wait complete. 2023-06-23T18:36:45.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:36:45.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.178+0000 7fa3f7e3c700 1 Processor -- start 2023-06-23T18:36:45.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.178+0000 7fa3f7e3c700 1 -- start start 2023-06-23T18:36:45.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.179+0000 7fa3f7e3c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f0156ae0 0x7fa3f015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.179+0000 7fa3f7e3c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa3f00cc060 0x7fa3f00cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.179+0000 7fa3f7e3c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f015daf0 con 0x7fa3f0071710 2023-06-23T18:36:45.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.179+0000 7fa3f7e3c700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f00cca10 con 0x7fa3f00cc060 2023-06-23T18:36:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.179+0000 7fa3f7e3c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f015a560 con 0x7fa3f0156ae0 2023-06-23T18:36:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.180+0000 7fa3f53d7700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa3f00cc060 0x7fa3f00cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.180+0000 7fa3f5bd8700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f0156ae0 0x7fa3f015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.180+0000 7fa3f63d9700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fa3f0071710 0x7fa3f0071b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:52382/0 (socket says 172.21.15.79:52382) 2023-06-23T18:36:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.180+0000 7fa3f63d9700 1 -- 172.21.15.79:0/2065720107 learned_addr learned my addr 172.21.15.79:0/2065720107 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.180+0000 7fa3f5bd8700 1 -- 172.21.15.79:0/2065720107 >> v2:172.21.15.155:3300/0 conn(0x7fa3f00cc060 msgr2=0x7fa3f00cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.180+0000 7fa3f5bd8700 1 --2- 172.21.15.79:0/2065720107 >> v2:172.21.15.155:3300/0 conn(0x7fa3f00cc060 0x7fa3f00cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.180+0000 7fa3f5bd8700 1 -- 172.21.15.79:0/2065720107 >> v1:172.21.15.155:6789/0 conn(0x7fa3f0071710 legacy=0x7fa3f0071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.180+0000 7fa3f5bd8700 1 -- 172.21.15.79:0/2065720107 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f0159950 con 0x7fa3f0156ae0 2023-06-23T18:36:45.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.180+0000 7fa3f4bd6700 1 -- 172.21.15.79:0/2065720107 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735966351 0 0) 0x7fa3f015daf0 con 0x7fa3f0071710 2023-06-23T18:36:45.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.180+0000 7fa3f53d7700 1 --2- 172.21.15.79:0/2065720107 >> v2:172.21.15.155:3300/0 conn(0x7fa3f00cc060 0x7fa3f00cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:45.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.180+0000 7fa3f5bd8700 1 --2- 172.21.15.79:0/2065720107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f0156ae0 0x7fa3f015d5b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009840 tx=0x7fa3e0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a1e6b29fda2cac4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:45.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.181+0000 7fa3f4bd6700 1 -- 172.21.15.79:0/2065720107 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0011830 con 0x7fa3f0156ae0 2023-06-23T18:36:45.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.181+0000 7fa3f4bd6700 1 -- 172.21.15.79:0/2065720107 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e0011990 con 0x7fa3f0156ae0 2023-06-23T18:36:45.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.181+0000 7fa3f4bd6700 1 -- 172.21.15.79:0/2065720107 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0011d20 con 0x7fa3f0156ae0 2023-06-23T18:36:45.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.181+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/2065720107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f0156ae0 msgr2=0x7fa3f015d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.181+0000 7fa3f7e3c700 1 --2- 172.21.15.79:0/2065720107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f0156ae0 0x7fa3f015d5b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009840 tx=0x7fa3e0009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:45.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.181+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/2065720107 shutdown_connections 2023-06-23T18:36:45.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.181+0000 7fa3f7e3c700 1 --2- 172.21.15.79:0/2065720107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f0156ae0 0x7fa3f015d5b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.181+0000 7fa3f7e3c700 1 --2- 172.21.15.79:0/2065720107 >> v2:172.21.15.155:3300/0 conn(0x7fa3f00cc060 0x7fa3f00cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.181+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/2065720107 >> 172.21.15.79:0/2065720107 conn(0x7fa3f006afa0 msgr2=0x7fa3f006b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:45.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.181+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/2065720107 shutdown_connections 2023-06-23T18:36:45.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.181+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/2065720107 wait complete. 2023-06-23T18:36:45.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.182+0000 7fa3f7e3c700 1 Processor -- start 2023-06-23T18:36:45.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.182+0000 7fa3f7e3c700 1 -- start start 2023-06-23T18:36:45.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f7e3c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3f0071710 0x7fa3f00c3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f7e3c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3f00cc060 0x7fa3f0149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f7e3c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f0156ae0 0x7fa3f014da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f7e3c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00aea70 con 0x7fa3f00cc060 2023-06-23T18:36:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f7e3c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00aece0 con 0x7fa3f0071710 2023-06-23T18:36:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f7e3c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00aefc0 con 0x7fa3f0156ae0 2023-06-23T18:36:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f53d7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3f00cc060 0x7fa3f0149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f53d7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3f00cc060 0x7fa3f0149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41076/0 (socket says 172.21.15.79:41076) 2023-06-23T18:36:45.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f53d7700 1 -- 172.21.15.79:0/984493031 learned_addr learned my addr 172.21.15.79:0/984493031 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:45.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f5bd8700 1 --2- 172.21.15.79:0/984493031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3f0071710 0x7fa3f00c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f63d9700 1 --2- 172.21.15.79:0/984493031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f0156ae0 0x7fa3f014da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f53d7700 1 -- 172.21.15.79:0/984493031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f0156ae0 msgr2=0x7fa3f014da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f53d7700 1 --2- 172.21.15.79:0/984493031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f0156ae0 0x7fa3f014da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f53d7700 1 -- 172.21.15.79:0/984493031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3f0071710 msgr2=0x7fa3f00c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.183+0000 7fa3f53d7700 1 --2- 172.21.15.79:0/984493031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3f0071710 0x7fa3f00c3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.184+0000 7fa3f53d7700 1 -- 172.21.15.79:0/984493031 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec0094a0 con 0x7fa3f00cc060 2023-06-23T18:36:45.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.184+0000 7fa3f53d7700 1 --2- 172.21.15.79:0/984493031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3f00cc060 0x7fa3f0149c40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00ebe0 tx=0x7fa3ec009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:45.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.184+0000 7fa3e67fc700 1 -- 172.21.15.79:0/984493031 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec00c910 con 0x7fa3f00cc060 2023-06-23T18:36:45.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.184+0000 7fa3e67fc700 1 -- 172.21.15.79:0/984493031 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ec00ca70 con 0x7fa3f00cc060 2023-06-23T18:36:45.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.184+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/984493031 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e00094a0 con 0x7fa3f00cc060 2023-06-23T18:36:45.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.184+0000 7fa3e67fc700 1 -- 172.21.15.79:0/984493031 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec00cde0 con 0x7fa3f00cc060 2023-06-23T18:36:45.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.184+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/984493031 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f00b54d0 con 0x7fa3f00cc060 2023-06-23T18:36:45.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.185+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/984493031 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f00672c0 con 0x7fa3f00cc060 2023-06-23T18:36:45.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.185+0000 7fa3e67fc700 1 -- 172.21.15.79:0/984493031 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa3ec013470 con 0x7fa3f00cc060 2023-06-23T18:36:45.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.186+0000 7fa3e67fc700 1 --2- 172.21.15.79:0/984493031 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa3d406ace0 0x7fa3d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.186+0000 7fa3f5bd8700 1 --2- 172.21.15.79:0/984493031 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa3d406ace0 0x7fa3d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.186+0000 7fa3e67fc700 1 -- 172.21.15.79:0/984493031 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa3ec089690 con 0x7fa3f00cc060 2023-06-23T18:36:45.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.186+0000 7fa3f5bd8700 1 --2- 172.21.15.79:0/984493031 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa3d406ace0 0x7fa3d406d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00090b0 tx=0x7fa3e001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:45.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.190+0000 7fa3e67fc700 1 -- 172.21.15.79:0/984493031 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa3ec054700 con 0x7fa3f00cc060 2023-06-23T18:36:45.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.375+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/984493031 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa3f0159390 con 0x7fa3f00cc060 2023-06-23T18:36:45.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.376+0000 7fa3e67fc700 1 -- 172.21.15.79:0/984493031 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa3ec057d20 con 0x7fa3f00cc060 2023-06-23T18:36:45.381 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:36:45.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.378+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/984493031 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa3d406ace0 msgr2=0x7fa3d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.378+0000 7fa3f7e3c700 1 --2- 172.21.15.79:0/984493031 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa3d406ace0 0x7fa3d406d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00090b0 tx=0x7fa3e001c040 comp rx=0 tx=0).stop 2023-06-23T18:36:45.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.378+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/984493031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3f00cc060 msgr2=0x7fa3f0149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.378+0000 7fa3f7e3c700 1 --2- 172.21.15.79:0/984493031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3f00cc060 0x7fa3f0149c40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00ebe0 tx=0x7fa3ec009ec0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.378+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/984493031 shutdown_connections 2023-06-23T18:36:45.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.378+0000 7fa3f7e3c700 1 --2- 172.21.15.79:0/984493031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f0156ae0 0x7fa3f014da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.378+0000 7fa3f7e3c700 1 --2- 172.21.15.79:0/984493031 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa3d406ace0 0x7fa3d406d190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.378+0000 7fa3f7e3c700 1 --2- 172.21.15.79:0/984493031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3f0071710 0x7fa3f00c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.378+0000 7fa3f7e3c700 1 --2- 172.21.15.79:0/984493031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3f00cc060 0x7fa3f0149c40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.378+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/984493031 >> 172.21.15.79:0/984493031 conn(0x7fa3f006afa0 msgr2=0x7fa3f00caee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:45.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.378+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/984493031 shutdown_connections 2023-06-23T18:36:45.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.378+0000 7fa3f7e3c700 1 -- 172.21.15.79:0/984493031 wait complete. 2023-06-23T18:36:45.395 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:36:45.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.512+0000 7f0ecc784700 1 Processor -- start 2023-06-23T18:36:45.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.512+0000 7f0ecc784700 1 -- start start 2023-06-23T18:36:45.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.513+0000 7f0ecc784700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0ec41569c0 0x7f0ec41608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.513+0000 7f0ecc784700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec40cdc90 con 0x7f0ec40cd440 2023-06-23T18:36:45.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.513+0000 7f0ecc784700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec415d0f0 con 0x7f0ec406a5a0 2023-06-23T18:36:45.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.513+0000 7f0ecc784700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec40cf570 con 0x7f0ec41569c0 2023-06-23T18:36:45.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.513+0000 7f0ec9d1f700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f0ec40cd440 0x7f0ec4069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44126/0 (socket says 172.21.15.79:44126) 2023-06-23T18:36:45.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.513+0000 7f0ec9d1f700 1 -- 172.21.15.79:0/2973490330 learned_addr learned my addr 172.21.15.79:0/2973490330 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:45.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.513+0000 7f0eca520700 1 --2- 172.21.15.79:0/2973490330 >> v2:172.21.15.155:3300/0 conn(0x7f0ec41569c0 0x7f0ec41608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.513+0000 7f0ec951e700 1 -- 172.21.15.79:0/2973490330 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431056005 0 0) 0x7f0ec0001110 con 0x7f0ec40cd440 2023-06-23T18:36:45.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec951e700 1 -- 172.21.15.79:0/2973490330 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eb00035b0 con 0x7f0ec40cd440 2023-06-23T18:36:45.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec951e700 1 -- 172.21.15.79:0/2973490330 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1543983630 0 0) 0x7f0eb00035b0 con 0x7f0ec40cd440 2023-06-23T18:36:45.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec951e700 1 -- 172.21.15.79:0/2973490330 >> v2:172.21.15.155:3300/0 conn(0x7f0ec41569c0 msgr2=0x7f0ec41608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:45.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec951e700 1 --2- 172.21.15.79:0/2973490330 >> v2:172.21.15.155:3300/0 conn(0x7f0ec41569c0 0x7f0ec41608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec951e700 1 -- 172.21.15.79:0/2973490330 >> v1:172.21.15.155:6789/0 conn(0x7f0ec406a5a0 legacy=0x7f0ec40b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec951e700 1 -- 172.21.15.79:0/2973490330 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec4156d90 con 0x7f0ec40cd440 2023-06-23T18:36:45.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec951e700 1 -- 172.21.15.79:0/2973490330 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f0ec0001800 con 0x7f0ec40cd440 2023-06-23T18:36:45.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec951e700 1 -- 172.21.15.79:0/2973490330 >> v1:172.21.15.79:6789/0 conn(0x7f0ec40cd440 legacy=0x7f0ec4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec951e700 1 --2- 172.21.15.79:0/2973490330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0eb0003a00 0x7f0eb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec951e700 1 --2- 172.21.15.79:0/2973490330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0eb00043f0 0x7f0eb00048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec951e700 1 --2- 172.21.15.79:0/2973490330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0eb00050b0 0x7f0eb000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec951e700 1 -- 172.21.15.79:0/2973490330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb000c000 con 0x7f0eb00050b0 2023-06-23T18:36:45.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec951e700 1 -- 172.21.15.79:0/2973490330 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb000c140 con 0x7f0eb00043f0 2023-06-23T18:36:45.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec951e700 1 -- 172.21.15.79:0/2973490330 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb000c300 con 0x7f0eb0003a00 2023-06-23T18:36:45.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ecad21700 1 --2- 172.21.15.79:0/2973490330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0eb00050b0 0x7f0eb000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0eca520700 1 --2- 172.21.15.79:0/2973490330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0eb0003a00 0x7f0eb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ec9d1f700 1 --2- 172.21.15.79:0/2973490330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0eb00043f0 0x7f0eb00048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ecad21700 1 -- 172.21.15.79:0/2973490330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0eb0003a00 msgr2=0x7f0eb0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:45.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ecad21700 1 --2- 172.21.15.79:0/2973490330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0eb0003a00 0x7f0eb0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ecad21700 1 -- 172.21.15.79:0/2973490330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0eb00043f0 msgr2=0x7f0eb00048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:45.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ecad21700 1 --2- 172.21.15.79:0/2973490330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0eb00043f0 0x7f0eb00048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ecad21700 1 -- 172.21.15.79:0/2973490330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb00035b0 con 0x7f0eb00050b0 2023-06-23T18:36:45.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ecad21700 1 -- 172.21.15.79:0/2973490330 reap_dead start 2023-06-23T18:36:45.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.514+0000 7f0ecad21700 1 --2- 172.21.15.79:0/2973490330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0eb00050b0 0x7f0eb000ba10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00c260 tx=0x7f0eb0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866cead317a82f6c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:45.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.515+0000 7f0ec951e700 1 -- 172.21.15.79:0/2973490330 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc001e70 con 0x7f0eb00050b0 2023-06-23T18:36:45.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.515+0000 7f0ec951e700 1 -- 172.21.15.79:0/2973490330 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc00ce00 con 0x7f0eb00050b0 2023-06-23T18:36:45.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.515+0000 7f0ec951e700 1 -- 172.21.15.79:0/2973490330 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc003c40 con 0x7f0eb00050b0 2023-06-23T18:36:45.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.515+0000 7f0ecc784700 1 -- 172.21.15.79:0/2973490330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0eb00050b0 msgr2=0x7f0eb000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.515+0000 7f0ecc784700 1 --2- 172.21.15.79:0/2973490330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0eb00050b0 0x7f0eb000ba10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00c260 tx=0x7f0eb0005880 comp rx=0 tx=0).stop 2023-06-23T18:36:45.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.515+0000 7f0ecc784700 1 -- 172.21.15.79:0/2973490330 shutdown_connections 2023-06-23T18:36:45.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.515+0000 7f0ecc784700 1 --2- 172.21.15.79:0/2973490330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0eb00050b0 0x7f0eb000ba10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.515+0000 7f0ecc784700 1 -- 172.21.15.79:0/2973490330 >> 172.21.15.79:0/2973490330 conn(0x7f0ec406afa0 msgr2=0x7f0ec406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:45.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.515+0000 7f0ecc784700 1 -- 172.21.15.79:0/2973490330 shutdown_connections 2023-06-23T18:36:45.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.515+0000 7f0ecc784700 1 -- 172.21.15.79:0/2973490330 wait complete. 2023-06-23T18:36:45.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.516+0000 7f0ecc784700 1 Processor -- start 2023-06-23T18:36:45.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.516+0000 7f0ecc784700 1 -- start start 2023-06-23T18:36:45.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0ecc784700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0eb00050b0 0x7f0ec40b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0ecc784700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ec40c8b10 0x7f0ec4155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0ecc784700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ec415d580 0x7f0ec415da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0ecc784700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec4156340 con 0x7f0eb00050b0 2023-06-23T18:36:45.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0ecc784700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec415df60 con 0x7f0ec40c8b10 2023-06-23T18:36:45.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0ecc784700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec40c9840 con 0x7f0ec415d580 2023-06-23T18:36:45.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0eca520700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0eb00050b0 0x7f0ec40b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0eca520700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0eb00050b0 0x7f0ec40b52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41094/0 (socket says 172.21.15.79:41094) 2023-06-23T18:36:45.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0eca520700 1 -- 172.21.15.79:0/3766214994 learned_addr learned my addr 172.21.15.79:0/3766214994 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:45.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0ec9d1f700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ec40c8b10 0x7f0ec4155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0ecad21700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ec415d580 0x7f0ec415da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0eca520700 1 -- 172.21.15.79:0/3766214994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ec415d580 msgr2=0x7f0ec415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0eca520700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ec415d580 0x7f0ec415da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0eca520700 1 -- 172.21.15.79:0/3766214994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ec40c8b10 msgr2=0x7f0ec4155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0eca520700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ec40c8b10 0x7f0ec4155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.517+0000 7f0eca520700 1 -- 172.21.15.79:0/3766214994 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec41f28c0 con 0x7f0eb00050b0 2023-06-23T18:36:45.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.518+0000 7f0ec9d1f700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ec40c8b10 0x7f0ec4155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:45.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.518+0000 7f0eca520700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0eb00050b0 0x7f0ec40b52e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40119a0 tx=0x7f0eb4011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:45.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.518+0000 7f0ebaffd700 1 -- 172.21.15.79:0/3766214994 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb400f940 con 0x7f0eb00050b0 2023-06-23T18:36:45.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.518+0000 7f0ebaffd700 1 -- 172.21.15.79:0/3766214994 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb400faa0 con 0x7f0eb00050b0 2023-06-23T18:36:45.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.518+0000 7f0ecc784700 1 -- 172.21.15.79:0/3766214994 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec41f1a50 con 0x7f0eb00050b0 2023-06-23T18:36:45.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.518+0000 7f0ebaffd700 1 -- 172.21.15.79:0/3766214994 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb400fe30 con 0x7f0eb00050b0 2023-06-23T18:36:45.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.518+0000 7f0ecc784700 1 -- 172.21.15.79:0/3766214994 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec41f1f90 con 0x7f0eb00050b0 2023-06-23T18:36:45.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.519+0000 7f0ecc784700 1 -- 172.21.15.79:0/3766214994 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec40672c0 con 0x7f0eb00050b0 2023-06-23T18:36:45.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.520+0000 7f0ebaffd700 1 -- 172.21.15.79:0/3766214994 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0eb401d400 con 0x7f0eb00050b0 2023-06-23T18:36:45.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.520+0000 7f0ebaffd700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0ea806ace0 0x7f0ea806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.520+0000 7f0ebaffd700 1 -- 172.21.15.79:0/3766214994 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0eb408a1f0 con 0x7f0eb00050b0 2023-06-23T18:36:45.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.521+0000 7f0ec9d1f700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0ea806ace0 0x7f0ea806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.521+0000 7f0ec9d1f700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0ea806ace0 0x7f0ea806d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0ec41f2c10 tx=0x7f0ec0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:45.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.524+0000 7f0ebaffd700 1 -- 172.21.15.79:0/3766214994 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0eb4055340 con 0x7f0eb00050b0 2023-06-23T18:36:45.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.709+0000 7f0ecc784700 1 -- 172.21.15.79:0/3766214994 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ec40ccb70 con 0x7f0eb00050b0 2023-06-23T18:36:45.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.709+0000 7f0ebaffd700 1 -- 172.21.15.79:0/3766214994 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0eb4058960 con 0x7f0eb00050b0 2023-06-23T18:36:45.714 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:36:45.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.711+0000 7f0ecc784700 1 -- 172.21.15.79:0/3766214994 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0ea806ace0 msgr2=0x7f0ea806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.711+0000 7f0ecc784700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0ea806ace0 0x7f0ea806d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0ec41f2c10 tx=0x7f0ec0009040 comp rx=0 tx=0).stop 2023-06-23T18:36:45.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.711+0000 7f0ecc784700 1 -- 172.21.15.79:0/3766214994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0eb00050b0 msgr2=0x7f0ec40b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.711+0000 7f0ecc784700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0eb00050b0 0x7f0ec40b52e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40119a0 tx=0x7f0eb4011d60 comp rx=0 tx=0).stop 2023-06-23T18:36:45.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.711+0000 7f0ecc784700 1 -- 172.21.15.79:0/3766214994 shutdown_connections 2023-06-23T18:36:45.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.711+0000 7f0ecc784700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ec415d580 0x7f0ec415da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.711+0000 7f0ecc784700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0eb00050b0 0x7f0ec40b52e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.711+0000 7f0ecc784700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0ea806ace0 0x7f0ea806d190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.711+0000 7f0ecc784700 1 --2- 172.21.15.79:0/3766214994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ec40c8b10 0x7f0ec4155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.711+0000 7f0ecc784700 1 -- 172.21.15.79:0/3766214994 >> 172.21.15.79:0/3766214994 conn(0x7f0ec406afa0 msgr2=0x7f0ec40c53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:45.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.711+0000 7f0ecc784700 1 -- 172.21.15.79:0/3766214994 shutdown_connections 2023-06-23T18:36:45.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.711+0000 7f0ecc784700 1 -- 172.21.15.79:0/3766214994 wait complete. 2023-06-23T18:36:45.728 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:36:45.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.842+0000 7fcc808fd700 1 Processor -- start 2023-06-23T18:36:45.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.842+0000 7fcc808fd700 1 -- start start 2023-06-23T18:36:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.843+0000 7fcc808fd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcc7c0cec00 0x7fcc7c1602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.843+0000 7fcc808fd700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcc7c0b17d0 0x7fcc7c160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.843+0000 7fcc808fd700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7c15fe30 con 0x7fcc7c15fa60 2023-06-23T18:36:45.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.843+0000 7fcc808fd700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7c06ce10 con 0x7fcc7c0cec00 2023-06-23T18:36:45.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.843+0000 7fcc808fd700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7c160d60 con 0x7fcc7c0b17d0 2023-06-23T18:36:45.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.843+0000 7fcc79d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcc7c0cec00 0x7fcc7c1602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.843+0000 7fcc79d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcc7c0cec00 0x7fcc7c1602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41106/0 (socket says 172.21.15.79:41106) 2023-06-23T18:36:45.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.843+0000 7fcc79d9b700 1 -- 172.21.15.79:0/2775159748 learned_addr learned my addr 172.21.15.79:0/2775159748 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:45.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.843+0000 7fcc7ad9d700 1 --2- 172.21.15.79:0/2775159748 >> v2:172.21.15.155:3300/0 conn(0x7fcc7c0b17d0 0x7fcc7c160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.843+0000 7fcc79d9b700 1 -- 172.21.15.79:0/2775159748 >> v2:172.21.15.155:3300/0 conn(0x7fcc7c0b17d0 msgr2=0x7fcc7c160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:45.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.843+0000 7fcc79d9b700 1 --2- 172.21.15.79:0/2775159748 >> v2:172.21.15.155:3300/0 conn(0x7fcc7c0b17d0 0x7fcc7c160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.843+0000 7fcc79d9b700 1 -- 172.21.15.79:0/2775159748 >> v1:172.21.15.155:6789/0 conn(0x7fcc7c15fa60 legacy=0x7fcc7c0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:45.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.843+0000 7fcc79d9b700 1 -- 172.21.15.79:0/2775159748 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c0b0a10 con 0x7fcc7c0cec00 2023-06-23T18:36:45.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.844+0000 7fcc79d9b700 1 --2- 172.21.15.79:0/2775159748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc7c0cec00 0x7fcc7c1602e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c06de50 tx=0x7fcc6c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad202d5382a7c91 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.844+0000 7fcc7959a700 1 -- 172.21.15.79:0/2775159748 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c00b870 con 0x7fcc7c0cec00 2023-06-23T18:36:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.844+0000 7fcc7959a700 1 -- 172.21.15.79:0/2775159748 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c00b9d0 con 0x7fcc7c0cec00 2023-06-23T18:36:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.844+0000 7fcc7959a700 1 -- 172.21.15.79:0/2775159748 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c00bd60 con 0x7fcc7c0cec00 2023-06-23T18:36:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.844+0000 7fcc808fd700 1 -- 172.21.15.79:0/2775159748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc7c0cec00 msgr2=0x7fcc7c1602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.844+0000 7fcc808fd700 1 --2- 172.21.15.79:0/2775159748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc7c0cec00 0x7fcc7c1602e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c06de50 tx=0x7fcc6c009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.844+0000 7fcc808fd700 1 -- 172.21.15.79:0/2775159748 shutdown_connections 2023-06-23T18:36:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.844+0000 7fcc808fd700 1 --2- 172.21.15.79:0/2775159748 >> v2:172.21.15.155:3300/0 conn(0x7fcc7c0b17d0 0x7fcc7c160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.844+0000 7fcc808fd700 1 --2- 172.21.15.79:0/2775159748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc7c0cec00 0x7fcc7c1602e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.844+0000 7fcc808fd700 1 -- 172.21.15.79:0/2775159748 >> 172.21.15.79:0/2775159748 conn(0x7fcc7c089fc0 msgr2=0x7fcc7c099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:45.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.844+0000 7fcc808fd700 1 -- 172.21.15.79:0/2775159748 shutdown_connections 2023-06-23T18:36:45.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.844+0000 7fcc808fd700 1 -- 172.21.15.79:0/2775159748 wait complete. 2023-06-23T18:36:45.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.846+0000 7fcc808fd700 1 Processor -- start 2023-06-23T18:36:45.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.846+0000 7fcc808fd700 1 -- start start 2023-06-23T18:36:45.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc808fd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc7c0b17d0 0x7fcc7c15aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc808fd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc7c0cec00 0x7fcc7c06b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc808fd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc7c15fa60 0x7fcc7c06bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc808fd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c15b400 con 0x7fcc7c15fa60 2023-06-23T18:36:45.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc808fd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c0adbb0 con 0x7fcc7c0cec00 2023-06-23T18:36:45.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc808fd700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c1e7b80 con 0x7fcc7c0b17d0 2023-06-23T18:36:45.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc7ad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc7c15fa60 0x7fcc7c06bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc7ad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc7c15fa60 0x7fcc7c06bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41112/0 (socket says 172.21.15.79:41112) 2023-06-23T18:36:45.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc7ad9d700 1 -- 172.21.15.79:0/263728790 learned_addr learned my addr 172.21.15.79:0/263728790 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:45.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc79d9b700 1 --2- 172.21.15.79:0/263728790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc7c0cec00 0x7fcc7c06b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc7a59c700 1 --2- 172.21.15.79:0/263728790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc7c0b17d0 0x7fcc7c15aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc7ad9d700 1 -- 172.21.15.79:0/263728790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc7c0b17d0 msgr2=0x7fcc7c15aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc7ad9d700 1 --2- 172.21.15.79:0/263728790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc7c0b17d0 0x7fcc7c15aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc7ad9d700 1 -- 172.21.15.79:0/263728790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc7c0cec00 msgr2=0x7fcc7c06b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:45.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.847+0000 7fcc7ad9d700 1 --2- 172.21.15.79:0/263728790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc7c0cec00 0x7fcc7c06b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:45.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.848+0000 7fcc7ad9d700 1 -- 172.21.15.79:0/263728790 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c009550 con 0x7fcc7c15fa60 2023-06-23T18:36:45.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.848+0000 7fcc7ad9d700 1 --2- 172.21.15.79:0/263728790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc7c15fa60 0x7fcc7c06bcb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009fd0 tx=0x7fcc7400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:45.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.848+0000 7fcc6affd700 1 -- 172.21.15.79:0/263728790 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74007ea0 con 0x7fcc7c15fa60 2023-06-23T18:36:45.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.848+0000 7fcc6affd700 1 -- 172.21.15.79:0/263728790 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7400ed30 con 0x7fcc7c15fa60 2023-06-23T18:36:45.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.848+0000 7fcc808fd700 1 -- 172.21.15.79:0/263728790 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7c1e7e60 con 0x7fcc7c15fa60 2023-06-23T18:36:45.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.848+0000 7fcc6affd700 1 -- 172.21.15.79:0/263728790 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7400eea0 con 0x7fcc7c15fa60 2023-06-23T18:36:45.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.848+0000 7fcc808fd700 1 -- 172.21.15.79:0/263728790 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7c0ade50 con 0x7fcc7c15fa60 2023-06-23T18:36:45.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.849+0000 7fcc808fd700 1 -- 172.21.15.79:0/263728790 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7c1e8380 con 0x7fcc7c15fa60 2023-06-23T18:36:45.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.849+0000 7fcc6affd700 1 -- 172.21.15.79:0/263728790 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcc74014660 con 0x7fcc7c15fa60 2023-06-23T18:36:45.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.850+0000 7fcc6affd700 1 --2- 172.21.15.79:0/263728790 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcc5c06ace0 0x7fcc5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:45.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.850+0000 7fcc6affd700 1 -- 172.21.15.79:0/263728790 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fcc74089aa0 con 0x7fcc7c15fa60 2023-06-23T18:36:45.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.850+0000 7fcc7a59c700 1 --2- 172.21.15.79:0/263728790 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcc5c06ace0 0x7fcc5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:45.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.850+0000 7fcc7a59c700 1 --2- 172.21.15.79:0/263728790 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcc5c06ace0 0x7fcc5c06d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc70000fb0 tx=0x7fcc70009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:45.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:45.854+0000 7fcc6affd700 1 -- 172.21.15.79:0/263728790 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcc7c1e8380 con 0x7fcc7c15fa60 2023-06-23T18:36:46.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.086+0000 7fcc808fd700 1 -- 172.21.15.79:0/263728790 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc7c1e8380 con 0x7fcc7c15fa60 2023-06-23T18:36:46.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.088+0000 7fcc6affd700 1 -- 172.21.15.79:0/263728790 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7fcc7c1e8380 con 0x7fcc7c15fa60 2023-06-23T18:36:46.093 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:46.093 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/399719238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":399719238},{"type":"v1","addr":"172.21.15.155:6834","nonce":399719238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1846440916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1846440916},{"type":"v1","addr":"172.21.15.155:6835","nonce":1846440916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3310180303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3310180303},{"type":"v1","addr":"172.21.15.79:6835","nonce":3310180303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:35:53.511495+0000","modified":"2023-06-23T18:35:56.527609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6837/3900137670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3900137670},{"type":"v1","addr":"172.21.15.79:6837","nonce":3900137670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:36:46.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.090+0000 7fcc808fd700 1 -- 172.21.15.79:0/263728790 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcc5c06ace0 msgr2=0x7fcc5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.090+0000 7fcc808fd700 1 --2- 172.21.15.79:0/263728790 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcc5c06ace0 0x7fcc5c06d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc70000fb0 tx=0x7fcc70009040 comp rx=0 tx=0).stop 2023-06-23T18:36:46.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.090+0000 7fcc808fd700 1 -- 172.21.15.79:0/263728790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc7c15fa60 msgr2=0x7fcc7c06bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.090+0000 7fcc808fd700 1 --2- 172.21.15.79:0/263728790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc7c15fa60 0x7fcc7c06bcb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009fd0 tx=0x7fcc7400e3c0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.090+0000 7fcc808fd700 1 -- 172.21.15.79:0/263728790 shutdown_connections 2023-06-23T18:36:46.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.090+0000 7fcc808fd700 1 --2- 172.21.15.79:0/263728790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc7c15fa60 0x7fcc7c06bcb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.090+0000 7fcc808fd700 1 --2- 172.21.15.79:0/263728790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc7c0b17d0 0x7fcc7c15aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.090+0000 7fcc808fd700 1 --2- 172.21.15.79:0/263728790 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcc5c06ace0 0x7fcc5c06d190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.090+0000 7fcc808fd700 1 --2- 172.21.15.79:0/263728790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc7c0cec00 0x7fcc7c06b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.090+0000 7fcc808fd700 1 -- 172.21.15.79:0/263728790 >> 172.21.15.79:0/263728790 conn(0x7fcc7c089fc0 msgr2=0x7fcc7c0ce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:46.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.090+0000 7fcc808fd700 1 -- 172.21.15.79:0/263728790 shutdown_connections 2023-06-23T18:36:46.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.090+0000 7fcc808fd700 1 -- 172.21.15.79:0/263728790 wait complete. 2023-06-23T18:36:46.098 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 8 2023-06-23T18:36:46.107 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:46.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.223+0000 7f9e09e91700 1 Processor -- start 2023-06-23T18:36:46.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.223+0000 7f9e09e91700 1 -- start start 2023-06-23T18:36:46.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.224+0000 7f9e09e91700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e04159170 0x7f9e040cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.224+0000 7f9e09e91700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0415cd50 con 0x7f9e040ce7f0 2023-06-23T18:36:46.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.224+0000 7f9e09e91700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0415cfa0 con 0x7f9e040cd900 2023-06-23T18:36:46.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.224+0000 7f9e09e91700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e040cdd70 con 0x7f9e04159170 2023-06-23T18:36:46.232 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.224+0000 7f9e03fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9e040ce7f0 0x7f9e040cc1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44162/0 (socket says 172.21.15.79:44162) 2023-06-23T18:36:46.232 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.224+0000 7f9e03fff700 1 -- 172.21.15.79:0/2994950930 learned_addr learned my addr 172.21.15.79:0/2994950930 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:46.232 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.224+0000 7f9e037fe700 1 --2- 172.21.15.79:0/2994950930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e04159170 0x7f9e040cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.232 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.224+0000 7f9e027fc700 1 -- 172.21.15.79:0/2994950930 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188099513 0 0) 0x7f9e0415cd50 con 0x7f9e040ce7f0 2023-06-23T18:36:46.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.224+0000 7f9e027fc700 1 -- 172.21.15.79:0/2994950930 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dec0035b0 con 0x7f9e040ce7f0 2023-06-23T18:36:46.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.224+0000 7f9e027fc700 1 -- 172.21.15.79:0/2994950930 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3306644066 0 0) 0x7f9dec0035b0 con 0x7f9e040ce7f0 2023-06-23T18:36:46.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.224+0000 7f9e027fc700 1 -- 172.21.15.79:0/2994950930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e04159170 msgr2=0x7f9e040cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:46.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.224+0000 7f9e027fc700 1 --2- 172.21.15.79:0/2994950930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e04159170 0x7f9e040cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.224+0000 7f9e027fc700 1 -- 172.21.15.79:0/2994950930 >> v1:172.21.15.155:6789/0 conn(0x7f9e040cd900 legacy=0x7f9e040ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.224+0000 7f9e027fc700 1 -- 172.21.15.79:0/2994950930 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0415d280 con 0x7f9e040ce7f0 2023-06-23T18:36:46.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e027fc700 1 -- 172.21.15.79:0/2994950930 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f9dfc0019f0 con 0x7f9e040ce7f0 2023-06-23T18:36:46.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e027fc700 1 -- 172.21.15.79:0/2994950930 >> v1:172.21.15.79:6789/0 conn(0x7f9e040ce7f0 legacy=0x7f9e040cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e027fc700 1 --2- 172.21.15.79:0/2994950930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e04159170 0x7f9dec003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e027fc700 1 --2- 172.21.15.79:0/2994950930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dec003fb0 0x7f9dec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e027fc700 1 --2- 172.21.15.79:0/2994950930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dec004d90 0x7f9dec005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e027fc700 1 -- 172.21.15.79:0/2994950930 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e040cdd70 con 0x7f9dec004d90 2023-06-23T18:36:46.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e027fc700 1 -- 172.21.15.79:0/2994950930 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec00bba0 con 0x7f9dec003fb0 2023-06-23T18:36:46.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e027fc700 1 -- 172.21.15.79:0/2994950930 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec00bd60 con 0x7f9e04159170 2023-06-23T18:36:46.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e03fff700 1 --2- 172.21.15.79:0/2994950930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dec004d90 0x7f9dec005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e037fe700 1 --2- 172.21.15.79:0/2994950930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e04159170 0x7f9dec003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e02ffd700 1 --2- 172.21.15.79:0/2994950930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dec003fb0 0x7f9dec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e03fff700 1 -- 172.21.15.79:0/2994950930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e04159170 msgr2=0x7f9dec003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:46.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e03fff700 1 --2- 172.21.15.79:0/2994950930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e04159170 0x7f9dec003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e03fff700 1 -- 172.21.15.79:0/2994950930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dec003fb0 msgr2=0x7f9dec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:46.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e03fff700 1 --2- 172.21.15.79:0/2994950930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dec003fb0 0x7f9dec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e03fff700 1 -- 172.21.15.79:0/2994950930 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec0035b0 con 0x7f9dec004d90 2023-06-23T18:36:46.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e03fff700 1 --2- 172.21.15.79:0/2994950930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dec004d90 0x7f9dec005200 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00a800 tx=0x7f9dec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ae02ade5acdbb2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:46.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e027fc700 1 -- 172.21.15.79:0/2994950930 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc00a2f0 con 0x7f9dec004d90 2023-06-23T18:36:46.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e027fc700 1 -- 172.21.15.79:0/2994950930 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dfc012600 con 0x7f9dec004d90 2023-06-23T18:36:46.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.225+0000 7f9e027fc700 1 -- 172.21.15.79:0/2994950930 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0129b0 con 0x7f9dec004d90 2023-06-23T18:36:46.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.226+0000 7f9e09e91700 1 -- 172.21.15.79:0/2994950930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dec004d90 msgr2=0x7f9dec005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.226+0000 7f9e09e91700 1 --2- 172.21.15.79:0/2994950930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dec004d90 0x7f9dec005200 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00a800 tx=0x7f9dec005880 comp rx=0 tx=0).stop 2023-06-23T18:36:46.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.226+0000 7f9e03fff700 1 -- 172.21.15.79:0/2994950930 reap_dead start 2023-06-23T18:36:46.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.226+0000 7f9e09e91700 1 -- 172.21.15.79:0/2994950930 shutdown_connections 2023-06-23T18:36:46.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.226+0000 7f9e09e91700 1 -- 172.21.15.79:0/2994950930 >> 172.21.15.79:0/2994950930 conn(0x7f9e0406d660 msgr2=0x7f9e0406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:46.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.226+0000 7f9e09e91700 1 -- 172.21.15.79:0/2994950930 shutdown_connections 2023-06-23T18:36:46.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.226+0000 7f9e09e91700 1 -- 172.21.15.79:0/2994950930 wait complete. 2023-06-23T18:36:46.239 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.227+0000 7f9e09e91700 1 Processor -- start 2023-06-23T18:36:46.239 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.227+0000 7f9e09e91700 1 -- start start 2023-06-23T18:36:46.239 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e09e91700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e0414cd60 0x7f9e040c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.239 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e09e91700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e040c68c0 0x7f9e040af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.240 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e09e91700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e040af670 0x7f9e0415b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.240 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e09e91700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e041f4510 con 0x7f9e0414cd60 2023-06-23T18:36:46.240 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e09e91700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e040afae0 con 0x7f9e040c68c0 2023-06-23T18:36:46.240 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e09e91700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e041f4080 con 0x7f9e040af670 2023-06-23T18:36:46.240 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e037fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e0414cd60 0x7f9e040c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.240 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e037fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e0414cd60 0x7f9e040c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41128/0 (socket says 172.21.15.79:41128) 2023-06-23T18:36:46.240 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e037fe700 1 -- 172.21.15.79:0/234732558 learned_addr learned my addr 172.21.15.79:0/234732558 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:46.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e02ffd700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e040c68c0 0x7f9e040af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e03fff700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e040af670 0x7f9e0415b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e037fe700 1 -- 172.21.15.79:0/234732558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e040af670 msgr2=0x7f9e0415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e037fe700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e040af670 0x7f9e0415b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e037fe700 1 -- 172.21.15.79:0/234732558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e040c68c0 msgr2=0x7f9e040af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e037fe700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e040c68c0 0x7f9e040af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e037fe700 1 -- 172.21.15.79:0/234732558 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e041f3790 con 0x7f9e0414cd60 2023-06-23T18:36:46.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e02ffd700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e040c68c0 0x7f9e040af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:46.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.228+0000 7f9e037fe700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e0414cd60 0x7f9e040c5e20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9df8011b00 tx=0x7f9df8011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:46.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.229+0000 7f9debfff700 1 -- 172.21.15.79:0/234732558 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df800f9a0 con 0x7f9e0414cd60 2023-06-23T18:36:46.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.229+0000 7f9debfff700 1 -- 172.21.15.79:0/234732558 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df800fb00 con 0x7f9e0414cd60 2023-06-23T18:36:46.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.229+0000 7f9e09e91700 1 -- 172.21.15.79:0/234732558 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e040c6e30 con 0x7f9e0414cd60 2023-06-23T18:36:46.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.229+0000 7f9debfff700 1 -- 172.21.15.79:0/234732558 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df800feb0 con 0x7f9e0414cd60 2023-06-23T18:36:46.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.229+0000 7f9e09e91700 1 -- 172.21.15.79:0/234732558 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e041f3d10 con 0x7f9e0414cd60 2023-06-23T18:36:46.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.230+0000 7f9e09e91700 1 -- 172.21.15.79:0/234732558 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e040cd6c0 con 0x7f9e0414cd60 2023-06-23T18:36:46.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.230+0000 7f9debfff700 1 -- 172.21.15.79:0/234732558 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9df8014440 con 0x7f9e0414cd60 2023-06-23T18:36:46.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.230+0000 7f9debfff700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9df006ace0 0x7f9df006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.231+0000 7f9e02ffd700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9df006ace0 0x7f9df006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.231+0000 7f9debfff700 1 -- 172.21.15.79:0/234732558 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9df8089b10 con 0x7f9e0414cd60 2023-06-23T18:36:46.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.231+0000 7f9e02ffd700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9df006ace0 0x7f9df006d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e040c5a80 tx=0x7f9df4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:46.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.235+0000 7f9debfff700 1 -- 172.21.15.79:0/234732558 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9df8058760 con 0x7f9e0414cd60 2023-06-23T18:36:46.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.420+0000 7f9e09e91700 1 -- 172.21.15.79:0/234732558 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e04157410 con 0x7f9e0414cd60 2023-06-23T18:36:46.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.421+0000 7f9debfff700 1 -- 172.21.15.79:0/234732558 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18329 (secure 0 0 0) 0x7f9df80582f0 con 0x7f9e0414cd60 2023-06-23T18:36:46.427 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:46.427 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":22,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:36:42.066263+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1a","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:46.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.423+0000 7f9e09e91700 1 -- 172.21.15.79:0/234732558 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9df006ace0 msgr2=0x7f9df006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.423+0000 7f9e09e91700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9df006ace0 0x7f9df006d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e040c5a80 tx=0x7f9df4009040 comp rx=0 tx=0).stop 2023-06-23T18:36:46.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.423+0000 7f9e09e91700 1 -- 172.21.15.79:0/234732558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e0414cd60 msgr2=0x7f9e040c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.423+0000 7f9e09e91700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e0414cd60 0x7f9e040c5e20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9df8011b00 tx=0x7f9df8011ec0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.423+0000 7f9e09e91700 1 -- 172.21.15.79:0/234732558 shutdown_connections 2023-06-23T18:36:46.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.423+0000 7f9e09e91700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e040af670 0x7f9e0415b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.423+0000 7f9e09e91700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e0414cd60 0x7f9e040c5e20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.423+0000 7f9e09e91700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9df006ace0 0x7f9df006d190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.423+0000 7f9e09e91700 1 --2- 172.21.15.79:0/234732558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e040c68c0 0x7f9e040af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.423+0000 7f9e09e91700 1 -- 172.21.15.79:0/234732558 >> 172.21.15.79:0/234732558 conn(0x7f9e0406d660 msgr2=0x7f9e041584a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:46.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.424+0000 7f9e09e91700 1 -- 172.21.15.79:0/234732558 shutdown_connections 2023-06-23T18:36:46.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.424+0000 7f9e09e91700 1 -- 172.21.15.79:0/234732558 wait complete. 2023-06-23T18:36:46.441 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-06-23T18:36:46.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:36:46.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.556+0000 7f2002854700 1 Processor -- start 2023-06-23T18:36:46.562 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.556+0000 7f2002854700 1 -- start start 2023-06-23T18:36:46.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.557+0000 7f2002854700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc070c30 0x7f1ffc15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.557+0000 7f2002854700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1ffc159bf0 0x7f1ffc0ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.557+0000 7f2002854700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc15db00 con 0x7f1ffc071000 2023-06-23T18:36:46.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.557+0000 7f2002854700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ffc15cbb0 con 0x7f1ffc159bf0 2023-06-23T18:36:46.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.557+0000 7f2002854700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc15a130 con 0x7f1ffc070c30 2023-06-23T18:36:46.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.558+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc070c30 0x7f1ffc15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.558+0000 7f2000df1700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1ffc159bf0 0x7f1ffc0ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.558+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc070c30 0x7f1ffc15d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:33906/0 (socket says 172.21.15.79:33906) 2023-06-23T18:36:46.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.558+0000 7f1ffbfff700 1 -- 172.21.15.79:0/3367145660 learned_addr learned my addr 172.21.15.79:0/3367145660 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:46.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.558+0000 7f1ffbfff700 1 -- 172.21.15.79:0/3367145660 >> v2:172.21.15.155:3300/0 conn(0x7f1ffc159bf0 msgr2=0x7f1ffc0ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:46.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.558+0000 7f1ffbfff700 1 --2- 172.21.15.79:0/3367145660 >> v2:172.21.15.155:3300/0 conn(0x7f1ffc159bf0 0x7f1ffc0ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.558+0000 7f1ffbfff700 1 -- 172.21.15.79:0/3367145660 >> v1:172.21.15.155:6789/0 conn(0x7f1ffc071000 legacy=0x7f1ffc1594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.565 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.558+0000 7f1ffbfff700 1 -- 172.21.15.79:0/3367145660 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc15a410 con 0x7f1ffc070c30 2023-06-23T18:36:46.565 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.558+0000 7f1ffbfff700 1 --2- 172.21.15.79:0/3367145660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc070c30 0x7f1ffc15d5c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00098f0 tx=0x7f1ff0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2d3b422a66311ce server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:46.565 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.558+0000 7f1ffaffd700 1 -- 172.21.15.79:0/3367145660 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0011830 con 0x7f1ffc070c30 2023-06-23T18:36:46.565 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.558+0000 7f1ffaffd700 1 -- 172.21.15.79:0/3367145660 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff0011990 con 0x7f1ffc070c30 2023-06-23T18:36:46.565 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.558+0000 7f1ffaffd700 1 -- 172.21.15.79:0/3367145660 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0011d20 con 0x7f1ffc070c30 2023-06-23T18:36:46.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.559+0000 7f2002854700 1 -- 172.21.15.79:0/3367145660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc070c30 msgr2=0x7f1ffc15d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.559+0000 7f2002854700 1 --2- 172.21.15.79:0/3367145660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc070c30 0x7f1ffc15d5c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00098f0 tx=0x7f1ff0009c00 comp rx=0 tx=0).stop 2023-06-23T18:36:46.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.559+0000 7f2002854700 1 -- 172.21.15.79:0/3367145660 shutdown_connections 2023-06-23T18:36:46.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.559+0000 7f2002854700 1 --2- 172.21.15.79:0/3367145660 >> v2:172.21.15.155:3300/0 conn(0x7f1ffc159bf0 0x7f1ffc0ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.559+0000 7f2002854700 1 --2- 172.21.15.79:0/3367145660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc070c30 0x7f1ffc15d5c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.559+0000 7f2002854700 1 -- 172.21.15.79:0/3367145660 >> 172.21.15.79:0/3367145660 conn(0x7f1ffc0ceb80 msgr2=0x7f1ffc0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:46.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.559+0000 7f2002854700 1 -- 172.21.15.79:0/3367145660 shutdown_connections 2023-06-23T18:36:46.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.559+0000 7f2002854700 1 -- 172.21.15.79:0/3367145660 wait complete. 2023-06-23T18:36:46.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.560+0000 7f2002854700 1 Processor -- start 2023-06-23T18:36:46.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.560+0000 7f2002854700 1 -- start start 2023-06-23T18:36:46.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.561+0000 7f2002854700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ffc070c30 0x7f1ffc14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.561+0000 7f2002854700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc071000 0x7f1ffc14f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.561+0000 7f2002854700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc159bf0 0x7f1ffc0c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.561+0000 7f2002854700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc0b06e0 con 0x7f1ffc159bf0 2023-06-23T18:36:46.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.561+0000 7f2002854700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc14b690 con 0x7f1ffc070c30 2023-06-23T18:36:46.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.561+0000 7f2002854700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc148dd0 con 0x7f1ffc071000 2023-06-23T18:36:46.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.561+0000 7f2000df1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc159bf0 0x7f1ffc0c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.561+0000 7f2000df1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc159bf0 0x7f1ffc0c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41146/0 (socket says 172.21.15.79:41146) 2023-06-23T18:36:46.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.561+0000 7f2000df1700 1 -- 172.21.15.79:0/2078394745 learned_addr learned my addr 172.21.15.79:0/2078394745 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:46.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.561+0000 7f1ffbfff700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ffc070c30 0x7f1ffc14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.561+0000 7f1ffb7fe700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc071000 0x7f1ffc14f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.562+0000 7f2000df1700 1 -- 172.21.15.79:0/2078394745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc071000 msgr2=0x7f1ffc14f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.562+0000 7f2000df1700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc071000 0x7f1ffc14f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.562+0000 7f2000df1700 1 -- 172.21.15.79:0/2078394745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ffc070c30 msgr2=0x7f1ffc14ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.562+0000 7f2000df1700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ffc070c30 0x7f1ffc14ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.562+0000 7f2000df1700 1 -- 172.21.15.79:0/2078394745 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff0009550 con 0x7f1ffc159bf0 2023-06-23T18:36:46.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.562+0000 7f1ffbfff700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ffc070c30 0x7f1ffc14ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:46.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.562+0000 7f2000df1700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc159bf0 0x7f1ffc0c6ad0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400efc0 tx=0x7f1ff400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:46.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.562+0000 7f1ff8ff9700 1 -- 172.21.15.79:0/2078394745 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4009ba0 con 0x7f1ffc159bf0 2023-06-23T18:36:46.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.562+0000 7f1ff8ff9700 1 -- 172.21.15.79:0/2078394745 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff400f040 con 0x7f1ffc159bf0 2023-06-23T18:36:46.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.562+0000 7f2002854700 1 -- 172.21.15.79:0/2078394745 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ffc0c77b0 con 0x7f1ffc159bf0 2023-06-23T18:36:46.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.562+0000 7f1ff8ff9700 1 -- 172.21.15.79:0/2078394745 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4014480 con 0x7f1ffc159bf0 2023-06-23T18:36:46.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.562+0000 7f2002854700 1 -- 172.21.15.79:0/2078394745 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ffc0c7d70 con 0x7f1ffc159bf0 2023-06-23T18:36:46.573 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.563+0000 7f2002854700 1 -- 172.21.15.79:0/2078394745 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ffc04e240 con 0x7f1ffc159bf0 2023-06-23T18:36:46.573 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.563+0000 7f1ff8ff9700 1 -- 172.21.15.79:0/2078394745 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1ff40145e0 con 0x7f1ffc159bf0 2023-06-23T18:36:46.573 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.564+0000 7f1ff8ff9700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1fe806ace0 0x7f1fe806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.573 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.564+0000 7f1ff8ff9700 1 -- 172.21.15.79:0/2078394745 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1ff4089e10 con 0x7f1ffc159bf0 2023-06-23T18:36:46.573 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.564+0000 7f1ffbfff700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1fe806ace0 0x7f1fe806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.564+0000 7f1ffbfff700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1fe806ace0 0x7f1fe806d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00098f0 tx=0x7f1ff0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:46.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.568+0000 7f1ff8ff9700 1 -- 172.21.15.79:0/2078394745 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1ff4054e80 con 0x7f1ffc159bf0 2023-06-23T18:36:46.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.797+0000 7f2002854700 1 -- 172.21.15.79:0/2078394745 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ffc070870 con 0x7f1ffc159bf0 2023-06-23T18:36:46.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.798+0000 7f1ff8ff9700 1 -- 172.21.15.79:0/2078394745 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f1ff4018090 con 0x7f1ffc159bf0 2023-06-23T18:36:46.803 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:46.803 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/399719238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":399719238},{"type":"v1","addr":"172.21.15.155:6834","nonce":399719238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1846440916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1846440916},{"type":"v1","addr":"172.21.15.155:6835","nonce":1846440916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3310180303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3310180303},{"type":"v1","addr":"172.21.15.79:6835","nonce":3310180303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:35:53.511495+0000","modified":"2023-06-23T18:35:56.527609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6837/3900137670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3900137670},{"type":"v1","addr":"172.21.15.79:6837","nonce":3900137670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:36:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.800+0000 7f2002854700 1 -- 172.21.15.79:0/2078394745 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1fe806ace0 msgr2=0x7f1fe806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.800+0000 7f2002854700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1fe806ace0 0x7f1fe806d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00098f0 tx=0x7f1ff0008030 comp rx=0 tx=0).stop 2023-06-23T18:36:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.800+0000 7f2002854700 1 -- 172.21.15.79:0/2078394745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc159bf0 msgr2=0x7f1ffc0c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.800+0000 7f2002854700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc159bf0 0x7f1ffc0c6ad0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400efc0 tx=0x7f1ff400c3a0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.800+0000 7f2002854700 1 -- 172.21.15.79:0/2078394745 shutdown_connections 2023-06-23T18:36:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.800+0000 7f2002854700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc159bf0 0x7f1ffc0c6ad0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.800+0000 7f2002854700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1fe806ace0 0x7f1fe806d190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.800+0000 7f2002854700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ffc070c30 0x7f1ffc14ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.800+0000 7f2002854700 1 --2- 172.21.15.79:0/2078394745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc071000 0x7f1ffc14f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.800+0000 7f2002854700 1 -- 172.21.15.79:0/2078394745 >> 172.21.15.79:0/2078394745 conn(0x7f1ffc0ceb80 msgr2=0x7f1ffc1594e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.800+0000 7f2002854700 1 -- 172.21.15.79:0/2078394745 shutdown_connections 2023-06-23T18:36:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.800+0000 7f2002854700 1 -- 172.21.15.79:0/2078394745 wait complete. 2023-06-23T18:36:46.807 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 8 2023-06-23T18:36:46.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:36:46.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.931+0000 7f21e3389700 1 Processor -- start 2023-06-23T18:36:46.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.932+0000 7f21e3389700 1 -- start start 2023-06-23T18:36:46.938 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.932+0000 7f21e3389700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f21dc159170 0x7f21dc0cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.938 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.932+0000 7f21e3389700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21dc0cd900 0x7f21dc0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.938 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.932+0000 7f21e3389700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21dc15cd50 con 0x7f21dc0ce620 2023-06-23T18:36:46.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.932+0000 7f21e3389700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21dc0cd1e0 con 0x7f21dc159170 2023-06-23T18:36:46.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.932+0000 7f21e3389700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc0cdda0 con 0x7f21dc0cd900 2023-06-23T18:36:46.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.933+0000 7f21e1125700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f21dc159170 0x7f21dc0cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.933+0000 7f21e0924700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21dc0cd900 0x7f21dc0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.933+0000 7f21e1926700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f21dc0ce620 0x7f21dc071490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:52464/0 (socket says 172.21.15.79:52464) 2023-06-23T18:36:46.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.933+0000 7f21e1926700 1 -- 172.21.15.79:0/2762880262 learned_addr learned my addr 172.21.15.79:0/2762880262 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:46.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.933+0000 7f21e1125700 1 -- 172.21.15.79:0/2762880262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21dc0cd900 msgr2=0x7f21dc0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:46.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.933+0000 7f21e1125700 1 --2- 172.21.15.79:0/2762880262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21dc0cd900 0x7f21dc0ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.933+0000 7f21e1125700 1 -- 172.21.15.79:0/2762880262 >> v1:172.21.15.155:6789/0 conn(0x7f21dc0ce620 legacy=0x7f21dc071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.933+0000 7f21e1125700 1 -- 172.21.15.79:0/2762880262 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc15cfa0 con 0x7f21dc159170 2023-06-23T18:36:46.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.933+0000 7f21d3fff700 1 -- 172.21.15.79:0/2762880262 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302934376 0 0) 0x7f21dc15cd50 con 0x7f21dc0ce620 2023-06-23T18:36:46.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.933+0000 7f21e0924700 1 --2- 172.21.15.79:0/2762880262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21dc0cd900 0x7f21dc0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:46.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.933+0000 7f21e1125700 1 --2- 172.21.15.79:0/2762880262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21dc159170 0x7f21dc0cd3c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009840 tx=0x7f21cc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9c9bb48ef2a5075 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:46.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.933+0000 7f21d3fff700 1 -- 172.21.15.79:0/2762880262 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc011830 con 0x7f21dc159170 2023-06-23T18:36:46.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.934+0000 7f21d3fff700 1 -- 172.21.15.79:0/2762880262 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc011990 con 0x7f21dc159170 2023-06-23T18:36:46.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.934+0000 7f21d3fff700 1 -- 172.21.15.79:0/2762880262 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc011d20 con 0x7f21dc159170 2023-06-23T18:36:46.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.934+0000 7f21e3389700 1 -- 172.21.15.79:0/2762880262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21dc159170 msgr2=0x7f21dc0cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.934+0000 7f21e3389700 1 --2- 172.21.15.79:0/2762880262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21dc159170 0x7f21dc0cd3c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009840 tx=0x7f21cc009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:46.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.934+0000 7f21e3389700 1 -- 172.21.15.79:0/2762880262 shutdown_connections 2023-06-23T18:36:46.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.934+0000 7f21e3389700 1 --2- 172.21.15.79:0/2762880262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21dc159170 0x7f21dc0cd3c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.934+0000 7f21e3389700 1 --2- 172.21.15.79:0/2762880262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21dc0cd900 0x7f21dc0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.934+0000 7f21e3389700 1 -- 172.21.15.79:0/2762880262 >> 172.21.15.79:0/2762880262 conn(0x7f21dc06d660 msgr2=0x7f21dc06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:46.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.934+0000 7f21e3389700 1 -- 172.21.15.79:0/2762880262 shutdown_connections 2023-06-23T18:36:46.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.934+0000 7f21e3389700 1 -- 172.21.15.79:0/2762880262 wait complete. 2023-06-23T18:36:46.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.935+0000 7f21e3389700 1 Processor -- start 2023-06-23T18:36:46.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.935+0000 7f21e3389700 1 -- start start 2023-06-23T18:36:46.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.936+0000 7f21e3389700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc0cd900 0x7f21dc0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.936+0000 7f21e3389700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21dc0ce620 0x7f21dc0c3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.936+0000 7f21e3389700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21dc159170 0x7f21dc152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.936+0000 7f21e3389700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc0c61b0 con 0x7f21dc0cd900 2023-06-23T18:36:46.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.936+0000 7f21e3389700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc153450 con 0x7f21dc159170 2023-06-23T18:36:46.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.936+0000 7f21e3389700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc152e40 con 0x7f21dc0ce620 2023-06-23T18:36:46.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.936+0000 7f21e1125700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc0cd900 0x7f21dc0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.936+0000 7f21e1125700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc0cd900 0x7f21dc0b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41156/0 (socket says 172.21.15.79:41156) 2023-06-23T18:36:46.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.936+0000 7f21e1125700 1 -- 172.21.15.79:0/522159095 learned_addr learned my addr 172.21.15.79:0/522159095 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:46.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.936+0000 7f21e0924700 1 --2- 172.21.15.79:0/522159095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21dc0ce620 0x7f21dc0c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.936+0000 7f21e1926700 1 --2- 172.21.15.79:0/522159095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21dc159170 0x7f21dc152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.936+0000 7f21e1125700 1 -- 172.21.15.79:0/522159095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21dc0ce620 msgr2=0x7f21dc0c3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.936+0000 7f21e1125700 1 --2- 172.21.15.79:0/522159095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21dc0ce620 0x7f21dc0c3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.937+0000 7f21e1125700 1 -- 172.21.15.79:0/522159095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21dc159170 msgr2=0x7f21dc152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:46.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.937+0000 7f21e1125700 1 --2- 172.21.15.79:0/522159095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21dc159170 0x7f21dc152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:46.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.937+0000 7f21e1125700 1 -- 172.21.15.79:0/522159095 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80094a0 con 0x7f21dc0cd900 2023-06-23T18:36:46.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.937+0000 7f21e1125700 1 --2- 172.21.15.79:0/522159095 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc0cd900 0x7f21dc0b26d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009b90 tx=0x7f21cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:46.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.937+0000 7f21d1ffb700 1 -- 172.21.15.79:0/522159095 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0036a0 con 0x7f21dc0cd900 2023-06-23T18:36:46.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.937+0000 7f21d1ffb700 1 -- 172.21.15.79:0/522159095 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc01e070 con 0x7f21dc0cd900 2023-06-23T18:36:46.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.937+0000 7f21e3389700 1 -- 172.21.15.79:0/522159095 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21cc0094a0 con 0x7f21dc0cd900 2023-06-23T18:36:46.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.937+0000 7f21d1ffb700 1 -- 172.21.15.79:0/522159095 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc003a60 con 0x7f21dc0cd900 2023-06-23T18:36:46.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.937+0000 7f21e3389700 1 -- 172.21.15.79:0/522159095 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21dc0c5af0 con 0x7f21dc0cd900 2023-06-23T18:36:46.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.938+0000 7f21e3389700 1 -- 172.21.15.79:0/522159095 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21dc0b1f10 con 0x7f21dc0cd900 2023-06-23T18:36:46.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.938+0000 7f21d1ffb700 1 -- 172.21.15.79:0/522159095 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f21cc003bc0 con 0x7f21dc0cd900 2023-06-23T18:36:46.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.939+0000 7f21d1ffb700 1 --2- 172.21.15.79:0/522159095 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f21c006ace0 0x7f21c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:46.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.939+0000 7f21d1ffb700 1 -- 172.21.15.79:0/522159095 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f21cc08a460 con 0x7f21dc0cd900 2023-06-23T18:36:46.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.939+0000 7f21e0924700 1 --2- 172.21.15.79:0/522159095 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f21c006ace0 0x7f21c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:46.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.939+0000 7f21e0924700 1 --2- 172.21.15.79:0/522159095 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f21c006ace0 0x7f21c006d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f21dc1f5960 tx=0x7f21d8009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:46.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:46.943+0000 7f21d1ffb700 1 -- 172.21.15.79:0/522159095 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f21cc055440 con 0x7f21dc0cd900 2023-06-23T18:36:47.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.175+0000 7f21e3389700 1 -- 172.21.15.79:0/522159095 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21dc0b5770 con 0x7f21dc0cd900 2023-06-23T18:36:47.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.175+0000 7f21d1ffb700 1 -- 172.21.15.79:0/522159095 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f21cc058a60 con 0x7f21dc0cd900 2023-06-23T18:36:47.330 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:47.330 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/399719238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":399719238},{"type":"v1","addr":"172.21.15.155:6834","nonce":399719238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1846440916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1846440916},{"type":"v1","addr":"172.21.15.155:6835","nonce":1846440916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3310180303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3310180303},{"type":"v1","addr":"172.21.15.79:6835","nonce":3310180303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:35:53.511495+0000","modified":"2023-06-23T18:35:56.527609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6837/3900137670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3900137670},{"type":"v1","addr":"172.21.15.79:6837","nonce":3900137670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:36:47.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.177+0000 7f21e3389700 1 -- 172.21.15.79:0/522159095 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f21c006ace0 msgr2=0x7f21c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:47.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.177+0000 7f21e3389700 1 --2- 172.21.15.79:0/522159095 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f21c006ace0 0x7f21c006d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f21dc1f5960 tx=0x7f21d8009210 comp rx=0 tx=0).stop 2023-06-23T18:36:47.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.177+0000 7f21e3389700 1 -- 172.21.15.79:0/522159095 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc0cd900 msgr2=0x7f21dc0b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:47.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.177+0000 7f21e3389700 1 --2- 172.21.15.79:0/522159095 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc0cd900 0x7f21dc0b26d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009b90 tx=0x7f21cc004750 comp rx=0 tx=0).stop 2023-06-23T18:36:47.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.178+0000 7f21e3389700 1 -- 172.21.15.79:0/522159095 shutdown_connections 2023-06-23T18:36:47.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.178+0000 7f21e3389700 1 --2- 172.21.15.79:0/522159095 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f21c006ace0 0x7f21c006d190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.178+0000 7f21e3389700 1 --2- 172.21.15.79:0/522159095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21dc159170 0x7f21dc152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.178+0000 7f21e3389700 1 --2- 172.21.15.79:0/522159095 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc0cd900 0x7f21dc0b26d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.178+0000 7f21e3389700 1 --2- 172.21.15.79:0/522159095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21dc0ce620 0x7f21dc0c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.178+0000 7f21e3389700 1 -- 172.21.15.79:0/522159095 >> 172.21.15.79:0/522159095 conn(0x7f21dc06d660 msgr2=0x7f21dc158380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:47.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.178+0000 7f21e3389700 1 -- 172.21.15.79:0/522159095 shutdown_connections 2023-06-23T18:36:47.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.178+0000 7f21e3389700 1 -- 172.21.15.79:0/522159095 wait complete. 2023-06-23T18:36:47.334 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 8 2023-06-23T18:36:47.334 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:47.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.450+0000 7f1b6354c700 1 Processor -- start 2023-06-23T18:36:47.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.450+0000 7f1b6354c700 1 -- start start 2023-06-23T18:36:47.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.451+0000 7f1b6354c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b5c159170 0x7f1b5c0cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.451+0000 7f1b6354c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1b5c0cd900 0x7f1b5c0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.451+0000 7f1b6354c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5c15cd50 con 0x7f1b5c0ce620 2023-06-23T18:36:47.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.451+0000 7f1b6354c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5c0cd1e0 con 0x7f1b5c0cd900 2023-06-23T18:36:47.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.451+0000 7f1b6354c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0cdd70 con 0x7f1b5c159170 2023-06-23T18:36:47.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.451+0000 7f1b60ae7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1b5c0cd900 0x7f1b5c0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:47.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.451+0000 7f1b61ae9700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f1b5c0ce620 0x7f1b5c071490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44210/0 (socket says 172.21.15.79:44210) 2023-06-23T18:36:47.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.451+0000 7f1b61ae9700 1 -- 172.21.15.79:0/909248695 learned_addr learned my addr 172.21.15.79:0/909248695 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:47.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.451+0000 7f1b60ae7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1b5c0cd900 0x7f1b5c0ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41164/0 (socket says 172.21.15.79:41164) 2023-06-23T18:36:47.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.451+0000 7f1b60ae7700 1 -- 172.21.15.79:0/909248695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b5c159170 msgr2=0x7f1b5c0cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:47.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.451+0000 7f1b60ae7700 1 --2- 172.21.15.79:0/909248695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b5c159170 0x7f1b5c0cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.451+0000 7f1b60ae7700 1 -- 172.21.15.79:0/909248695 >> v1:172.21.15.79:6789/0 conn(0x7f1b5c0ce620 legacy=0x7f1b5c071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:47.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.451+0000 7f1b60ae7700 1 -- 172.21.15.79:0/909248695 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c15cfa0 con 0x7f1b5c0cd900 2023-06-23T18:36:47.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.451+0000 7f1b53fff700 1 -- 172.21.15.79:0/909248695 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841349908 0 0) 0x7f1b5c15cd50 con 0x7f1b5c0ce620 2023-06-23T18:36:47.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.452+0000 7f1b60ae7700 1 --2- 172.21.15.79:0/909248695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b5c0cd900 0x7f1b5c0ce0e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800bb10 tx=0x7f1b5800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376e90ac99fed385 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:47.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.452+0000 7f1b53fff700 1 -- 172.21.15.79:0/909248695 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5800dcf0 con 0x7f1b5c0cd900 2023-06-23T18:36:47.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.452+0000 7f1b53fff700 1 -- 172.21.15.79:0/909248695 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5800de50 con 0x7f1b5c0cd900 2023-06-23T18:36:47.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.452+0000 7f1b53fff700 1 -- 172.21.15.79:0/909248695 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58010690 con 0x7f1b5c0cd900 2023-06-23T18:36:47.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.452+0000 7f1b6354c700 1 -- 172.21.15.79:0/909248695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b5c0cd900 msgr2=0x7f1b5c0ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:47.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.452+0000 7f1b6354c700 1 --2- 172.21.15.79:0/909248695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b5c0cd900 0x7f1b5c0ce0e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800bb10 tx=0x7f1b5800be20 comp rx=0 tx=0).stop 2023-06-23T18:36:47.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.452+0000 7f1b6354c700 1 -- 172.21.15.79:0/909248695 shutdown_connections 2023-06-23T18:36:47.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.452+0000 7f1b6354c700 1 --2- 172.21.15.79:0/909248695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b5c159170 0x7f1b5c0cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.452+0000 7f1b6354c700 1 --2- 172.21.15.79:0/909248695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b5c0cd900 0x7f1b5c0ce0e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.452+0000 7f1b6354c700 1 -- 172.21.15.79:0/909248695 >> 172.21.15.79:0/909248695 conn(0x7f1b5c06d660 msgr2=0x7f1b5c06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:47.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.452+0000 7f1b6354c700 1 -- 172.21.15.79:0/909248695 shutdown_connections 2023-06-23T18:36:47.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.452+0000 7f1b6354c700 1 -- 172.21.15.79:0/909248695 wait complete. 2023-06-23T18:36:47.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.454+0000 7f1b6354c700 1 Processor -- start 2023-06-23T18:36:47.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.454+0000 7f1b6354c700 1 -- start start 2023-06-23T18:36:47.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.454+0000 7f1b6354c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b5c0cd900 0x7f1b5c15bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.454+0000 7f1b6354c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b5c0ce620 0x7f1b5c14ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.454+0000 7f1b6354c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b5c159170 0x7f1b5c0c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.454+0000 7f1b6354c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0cbaa0 con 0x7f1b5c0cd900 2023-06-23T18:36:47.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.454+0000 7f1b6354c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0c4770 con 0x7f1b5c159170 2023-06-23T18:36:47.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.454+0000 7f1b6354c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0c4a50 con 0x7f1b5c0ce620 2023-06-23T18:36:47.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.454+0000 7f1b612e8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b5c0cd900 0x7f1b5c15bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:47.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.454+0000 7f1b612e8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b5c0cd900 0x7f1b5c15bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41170/0 (socket says 172.21.15.79:41170) 2023-06-23T18:36:47.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.454+0000 7f1b612e8700 1 -- 172.21.15.79:0/2489291360 learned_addr learned my addr 172.21.15.79:0/2489291360 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:47.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.455+0000 7f1b612e8700 1 -- 172.21.15.79:0/2489291360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b5c0ce620 msgr2=0x7f1b5c14ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:47.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.455+0000 7f1b60ae7700 1 --2- 172.21.15.79:0/2489291360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b5c0ce620 0x7f1b5c14ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:47.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.455+0000 7f1b61ae9700 1 --2- 172.21.15.79:0/2489291360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b5c159170 0x7f1b5c0c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:47.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.455+0000 7f1b612e8700 1 --2- 172.21.15.79:0/2489291360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b5c0ce620 0x7f1b5c14ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.455+0000 7f1b612e8700 1 -- 172.21.15.79:0/2489291360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b5c159170 msgr2=0x7f1b5c0c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:47.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.455+0000 7f1b612e8700 1 --2- 172.21.15.79:0/2489291360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b5c159170 0x7f1b5c0c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.455+0000 7f1b612e8700 1 -- 172.21.15.79:0/2489291360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5800b740 con 0x7f1b5c0cd900 2023-06-23T18:36:47.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.455+0000 7f1b612e8700 1 --2- 172.21.15.79:0/2489291360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b5c0cd900 0x7f1b5c15bf40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00b4a0 tx=0x7f1b4c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:47.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.455+0000 7f1b51ffb700 1 -- 172.21.15.79:0/2489291360 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c011830 con 0x7f1b5c0cd900 2023-06-23T18:36:47.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.455+0000 7f1b51ffb700 1 -- 172.21.15.79:0/2489291360 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4c011990 con 0x7f1b5c0cd900 2023-06-23T18:36:47.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.455+0000 7f1b6354c700 1 -- 172.21.15.79:0/2489291360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c0c4d30 con 0x7f1b5c0cd900 2023-06-23T18:36:47.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.455+0000 7f1b51ffb700 1 -- 172.21.15.79:0/2489291360 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c011d40 con 0x7f1b5c0cd900 2023-06-23T18:36:47.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.455+0000 7f1b6354c700 1 -- 172.21.15.79:0/2489291360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c0c52f0 con 0x7f1b5c0cd900 2023-06-23T18:36:47.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.456+0000 7f1b6354c700 1 -- 172.21.15.79:0/2489291360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5c06ad80 con 0x7f1b5c0cd900 2023-06-23T18:36:47.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.456+0000 7f1b51ffb700 1 -- 172.21.15.79:0/2489291360 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1b4c01a400 con 0x7f1b5c0cd900 2023-06-23T18:36:47.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.457+0000 7f1b51ffb700 1 --2- 172.21.15.79:0/2489291360 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1b4006ace0 0x7f1b4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.457+0000 7f1b51ffb700 1 -- 172.21.15.79:0/2489291360 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1b4c0894b0 con 0x7f1b5c0cd900 2023-06-23T18:36:47.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.457+0000 7f1b60ae7700 1 --2- 172.21.15.79:0/2489291360 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1b4006ace0 0x7f1b4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:47.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.457+0000 7f1b60ae7700 1 --2- 172.21.15.79:0/2489291360 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1b4006ace0 0x7f1b4006d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0c2fa0 tx=0x7f1b5801b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:47.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.461+0000 7f1b51ffb700 1 -- 172.21.15.79:0/2489291360 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1b4c054530 con 0x7f1b5c0cd900 2023-06-23T18:36:47.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.647+0000 7f1b6354c700 1 -- 172.21.15.79:0/2489291360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b5c0b5770 con 0x7f1b5c0cd900 2023-06-23T18:36:47.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.648+0000 7f1b51ffb700 1 -- 172.21.15.79:0/2489291360 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18329 (secure 0 0 0) 0x7f1b4c057b50 con 0x7f1b5c0cd900 2023-06-23T18:36:47.654 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:47.655 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":22,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:36:42.066263+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:35:51.125894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:35:52.478806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1a","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:47.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.650+0000 7f1b6354c700 1 -- 172.21.15.79:0/2489291360 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1b4006ace0 msgr2=0x7f1b4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:47.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.650+0000 7f1b6354c700 1 --2- 172.21.15.79:0/2489291360 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1b4006ace0 0x7f1b4006d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0c2fa0 tx=0x7f1b5801b040 comp rx=0 tx=0).stop 2023-06-23T18:36:47.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.650+0000 7f1b6354c700 1 -- 172.21.15.79:0/2489291360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b5c0cd900 msgr2=0x7f1b5c15bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:47.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.650+0000 7f1b6354c700 1 --2- 172.21.15.79:0/2489291360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b5c0cd900 0x7f1b5c15bf40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00b4a0 tx=0x7f1b4c00b860 comp rx=0 tx=0).stop 2023-06-23T18:36:47.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.650+0000 7f1b6354c700 1 -- 172.21.15.79:0/2489291360 shutdown_connections 2023-06-23T18:36:47.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.650+0000 7f1b6354c700 1 --2- 172.21.15.79:0/2489291360 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1b4006ace0 0x7f1b4006d190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.650+0000 7f1b6354c700 1 --2- 172.21.15.79:0/2489291360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b5c159170 0x7f1b5c0c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.650+0000 7f1b6354c700 1 --2- 172.21.15.79:0/2489291360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b5c0cd900 0x7f1b5c15bf40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.650+0000 7f1b6354c700 1 --2- 172.21.15.79:0/2489291360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b5c0ce620 0x7f1b5c14ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.650+0000 7f1b6354c700 1 -- 172.21.15.79:0/2489291360 >> 172.21.15.79:0/2489291360 conn(0x7f1b5c06d660 msgr2=0x7f1b5c155ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:47.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.650+0000 7f1b6354c700 1 -- 172.21.15.79:0/2489291360 shutdown_connections 2023-06-23T18:36:47.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.650+0000 7f1b6354c700 1 -- 172.21.15.79:0/2489291360 wait complete. 2023-06-23T18:36:47.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-06-23T18:36:47.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.783+0000 7fd0ad3ce700 1 Processor -- start 2023-06-23T18:36:47.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.783+0000 7fd0ad3ce700 1 -- start start 2023-06-23T18:36:47.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.784+0000 7fd0ad3ce700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a80b1fd0 0x7fd0a80cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.784+0000 7fd0ad3ce700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd0a80b59c0 0x7fd0a815fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.784+0000 7fd0ad3ce700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a80d1460 con 0x7fd0a80d2690 2023-06-23T18:36:47.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.784+0000 7fd0ad3ce700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a80d2530 con 0x7fd0a80b59c0 2023-06-23T18:36:47.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.784+0000 7fd0ad3ce700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a80d1740 con 0x7fd0a80b1fd0 2023-06-23T18:36:47.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.784+0000 7fd0a67fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd0a80d2690 0x7fd0a80cf6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44222/0 (socket says 172.21.15.79:44222) 2023-06-23T18:36:47.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.784+0000 7fd0a67fc700 1 -- 172.21.15.79:0/1735366365 learned_addr learned my addr 172.21.15.79:0/1735366365 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:47.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.784+0000 7fd0a6ffd700 1 --2- 172.21.15.79:0/1735366365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a80b1fd0 0x7fd0a80cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:47.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.784+0000 7fd0a77fe700 1 --2- 172.21.15.79:0/1735366365 >> v2:172.21.15.155:3300/0 conn(0x7fd0a80b59c0 0x7fd0a815fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:47.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.784+0000 7fd0a5ffb700 1 -- 172.21.15.79:0/1735366365 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076527789 0 0) 0x7fd0a80d1460 con 0x7fd0a80d2690 2023-06-23T18:36:47.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.784+0000 7fd0a5ffb700 1 -- 172.21.15.79:0/1735366365 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0900035b0 con 0x7fd0a80d2690 2023-06-23T18:36:47.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.784+0000 7fd0a5ffb700 1 -- 172.21.15.79:0/1735366365 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1737863948 0 0) 0x7fd0900035b0 con 0x7fd0a80d2690 2023-06-23T18:36:47.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.784+0000 7fd0a5ffb700 1 -- 172.21.15.79:0/1735366365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a80b1fd0 msgr2=0x7fd0a80cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:47.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.784+0000 7fd0a5ffb700 1 --2- 172.21.15.79:0/1735366365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a80b1fd0 0x7fd0a80cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a5ffb700 1 -- 172.21.15.79:0/1735366365 >> v2:172.21.15.155:3300/0 conn(0x7fd0a80b59c0 msgr2=0x7fd0a815fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:47.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a5ffb700 1 --2- 172.21.15.79:0/1735366365 >> v2:172.21.15.155:3300/0 conn(0x7fd0a80b59c0 0x7fd0a815fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a5ffb700 1 -- 172.21.15.79:0/1735366365 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a80cd260 con 0x7fd0a80d2690 2023-06-23T18:36:47.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a5ffb700 1 -- 172.21.15.79:0/1735366365 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fd098001670 con 0x7fd0a80d2690 2023-06-23T18:36:47.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a5ffb700 1 -- 172.21.15.79:0/1735366365 >> v1:172.21.15.79:6789/0 conn(0x7fd0a80d2690 legacy=0x7fd0a80cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:47.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a5ffb700 1 --2- 172.21.15.79:0/1735366365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090003a00 0x7fd090003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a5ffb700 1 --2- 172.21.15.79:0/1735366365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd090004430 0x7fd0900048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a5ffb700 1 --2- 172.21.15.79:0/1735366365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a80b1fd0 0x7fd090005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a5ffb700 1 -- 172.21.15.79:0/1735366365 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a80d2530 con 0x7fd090003a00 2023-06-23T18:36:47.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a5ffb700 1 -- 172.21.15.79:0/1735366365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a80d1740 con 0x7fd090004430 2023-06-23T18:36:47.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a5ffb700 1 -- 172.21.15.79:0/1735366365 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09000ba10 con 0x7fd0a80b1fd0 2023-06-23T18:36:47.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a6ffd700 1 --2- 172.21.15.79:0/1735366365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090003a00 0x7fd090003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:47.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a77fe700 1 --2- 172.21.15.79:0/1735366365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a80b1fd0 0x7fd090005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:47.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a67fc700 1 --2- 172.21.15.79:0/1735366365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd090004430 0x7fd0900048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:47.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a6ffd700 1 -- 172.21.15.79:0/1735366365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a80b1fd0 msgr2=0x7fd090005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:47.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a6ffd700 1 --2- 172.21.15.79:0/1735366365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a80b1fd0 0x7fd090005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a6ffd700 1 -- 172.21.15.79:0/1735366365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd090004430 msgr2=0x7fd0900048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:47.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a6ffd700 1 --2- 172.21.15.79:0/1735366365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd090004430 0x7fd0900048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a6ffd700 1 -- 172.21.15.79:0/1735366365 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0900035b0 con 0x7fd090003a00 2023-06-23T18:36:47.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a6ffd700 1 --2- 172.21.15.79:0/1735366365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090003a00 0x7fd090003e10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009b90 tx=0x7fd090005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e062eb927b278680 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:47.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a5ffb700 1 -- 172.21.15.79:0/1735366365 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c00e6c0 con 0x7fd090003a00 2023-06-23T18:36:47.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a5ffb700 1 -- 172.21.15.79:0/1735366365 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c00e820 con 0x7fd090003a00 2023-06-23T18:36:47.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.785+0000 7fd0a5ffb700 1 -- 172.21.15.79:0/1735366365 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c00eb90 con 0x7fd090003a00 2023-06-23T18:36:47.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.786+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1735366365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090003a00 msgr2=0x7fd090003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:47.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.786+0000 7fd0ad3ce700 1 --2- 172.21.15.79:0/1735366365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090003a00 0x7fd090003e10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009b90 tx=0x7fd090005880 comp rx=0 tx=0).stop 2023-06-23T18:36:47.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.786+0000 7fd0a77fe700 1 -- 172.21.15.79:0/1735366365 reap_dead start 2023-06-23T18:36:47.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.786+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1735366365 shutdown_connections 2023-06-23T18:36:47.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.786+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1735366365 >> 172.21.15.79:0/1735366365 conn(0x7fd0a806b340 msgr2=0x7fd0a806b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:47.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.786+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1735366365 shutdown_connections 2023-06-23T18:36:47.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.786+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1735366365 wait complete. 2023-06-23T18:36:47.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.787+0000 7fd0ad3ce700 1 Processor -- start 2023-06-23T18:36:47.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.787+0000 7fd0ad3ce700 1 -- start start 2023-06-23T18:36:47.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0ad3ce700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090003a00 0x7fd0a815dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0ad3ce700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a80c7bd0 0x7fd0a80cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0ad3ce700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a8154fe0 0x7fd0a8155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0ad3ce700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a80c90a0 con 0x7fd090003a00 2023-06-23T18:36:47.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0ad3ce700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a81559a0 con 0x7fd0a8154fe0 2023-06-23T18:36:47.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0ad3ce700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a80c8420 con 0x7fd0a80c7bd0 2023-06-23T18:36:47.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0a6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090003a00 0x7fd0a815dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:47.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0a6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090003a00 0x7fd0a815dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41192/0 (socket says 172.21.15.79:41192) 2023-06-23T18:36:47.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0a6ffd700 1 -- 172.21.15.79:0/1806115341 learned_addr learned my addr 172.21.15.79:0/1806115341 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:47.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0a77fe700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a8154fe0 0x7fd0a8155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:47.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0a67fc700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a80c7bd0 0x7fd0a80cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:47.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0a6ffd700 1 -- 172.21.15.79:0/1806115341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a80c7bd0 msgr2=0x7fd0a80cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:47.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0a6ffd700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a80c7bd0 0x7fd0a80cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0a6ffd700 1 -- 172.21.15.79:0/1806115341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a8154fe0 msgr2=0x7fd0a8155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:47.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0a6ffd700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a8154fe0 0x7fd0a8155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:47.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0a6ffd700 1 -- 172.21.15.79:0/1806115341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a80c8700 con 0x7fd090003a00 2023-06-23T18:36:47.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0a77fe700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a8154fe0 0x7fd0a8155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:47.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.788+0000 7fd0a6ffd700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090003a00 0x7fd0a815dcd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0090b0 tx=0x7fd09c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:47.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.789+0000 7fd08f7fe700 1 -- 172.21.15.79:0/1806115341 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c00e6c0 con 0x7fd090003a00 2023-06-23T18:36:47.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.789+0000 7fd08f7fe700 1 -- 172.21.15.79:0/1806115341 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c003af0 con 0x7fd090003a00 2023-06-23T18:36:47.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.789+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1806115341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a80c89e0 con 0x7fd090003a00 2023-06-23T18:36:47.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.789+0000 7fd08f7fe700 1 -- 172.21.15.79:0/1806115341 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c003ea0 con 0x7fd090003a00 2023-06-23T18:36:47.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.789+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1806115341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a80c94c0 con 0x7fd090003a00 2023-06-23T18:36:47.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.789+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1806115341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a80d1110 con 0x7fd090003a00 2023-06-23T18:36:47.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.790+0000 7fd08f7fe700 1 -- 172.21.15.79:0/1806115341 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd09c013470 con 0x7fd090003a00 2023-06-23T18:36:47.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.790+0000 7fd08f7fe700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd09406ace0 0x7fd09406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:47.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.791+0000 7fd0a67fc700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd09406ace0 0x7fd09406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:47.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.791+0000 7fd08f7fe700 1 -- 172.21.15.79:0/1806115341 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd09c08afc0 con 0x7fd090003a00 2023-06-23T18:36:47.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.791+0000 7fd0a67fc700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd09406ace0 0x7fd09406d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd098001050 tx=0x7fd098009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:47.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:47.795+0000 7fd08f7fe700 1 -- 172.21.15.79:0/1806115341 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd09c0bd310 con 0x7fd090003a00 2023-06-23T18:36:48.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.022+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1806115341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd0a80b2f50 con 0x7fd090003a00 2023-06-23T18:36:48.097 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-06-23T18:36:48.092+0000 7f71a235a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-06-23T18:36:48.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.102+0000 7fd08f7fe700 1 -- 172.21.15.79:0/1806115341 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd09c017090 con 0x7fd090003a00 2023-06-23T18:36:48.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.104+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1806115341 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd09406ace0 msgr2=0x7fd09406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:48.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.104+0000 7fd0ad3ce700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd09406ace0 0x7fd09406d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd098001050 tx=0x7fd098009040 comp rx=0 tx=0).stop 2023-06-23T18:36:48.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.104+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1806115341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090003a00 msgr2=0x7fd0a815dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:48.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.104+0000 7fd0ad3ce700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090003a00 0x7fd0a815dcd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0090b0 tx=0x7fd09c0047c0 comp rx=0 tx=0).stop 2023-06-23T18:36:48.110 INFO:tasks.ceph.mds.a.smithi079.stderr: -3020> 2023-06-23T18:35:59.105+0000 7fd70b612700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-23T18:36:48.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.104+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1806115341 shutdown_connections 2023-06-23T18:36:48.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.104+0000 7fd0ad3ce700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd09406ace0 0x7fd09406d190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:48.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.104+0000 7fd0ad3ce700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a8154fe0 0x7fd0a8155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:48.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.104+0000 7fd0ad3ce700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd090003a00 0x7fd0a815dcd0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:48.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.104+0000 7fd0ad3ce700 1 --2- 172.21.15.79:0/1806115341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a80c7bd0 0x7fd0a80cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:48.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.104+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1806115341 >> 172.21.15.79:0/1806115341 conn(0x7fd0a806b340 msgr2=0x7fd0a80b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:48.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.104+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1806115341 shutdown_connections 2023-06-23T18:36:48.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.104+0000 7fd0ad3ce700 1 -- 172.21.15.79:0/1806115341 wait complete. 2023-06-23T18:36:48.111 INFO:teuthology.orchestra.run.smithi079.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-06-23T18:36:48.122 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-06-23T18:36:48.138 INFO:tasks.ceph.mds.a.smithi079.stdout:starting mds.a at 2023-06-23T18:36:48.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.240+0000 7f2fe262f700 1 Processor -- start 2023-06-23T18:36:48.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.240+0000 7f2fe262f700 1 -- start start 2023-06-23T18:36:48.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fe262f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fdc156ae0 0x7f2fdc15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:48.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fe262f700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2fdc0cc060 0x7f2fdc0cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:48.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fe262f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2fdc071710 0x7f2fdc071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:48.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fe262f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fdc0cca10 con 0x7f2fdc071710 2023-06-23T18:36:48.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fe262f700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fdc15daf0 con 0x7f2fdc0cc060 2023-06-23T18:36:48.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fe262f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc15a560 con 0x7f2fdc156ae0 2023-06-23T18:36:48.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fe0bcc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2fdc071710 0x7f2fdc071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:48.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fe0bcc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2fdc071710 0x7f2fdc071b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41214/0 (socket says 172.21.15.79:41214) 2023-06-23T18:36:48.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fe0bcc700 1 -- 172.21.15.79:0/1898758491 learned_addr learned my addr 172.21.15.79:0/1898758491 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:48.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fdbfff700 1 --2- 172.21.15.79:0/1898758491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fdc156ae0 0x7f2fdc15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:48.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fdb7fe700 1 --2- 172.21.15.79:0/1898758491 >> v2:172.21.15.155:3300/0 conn(0x7f2fdc0cc060 0x7f2fdc0cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:48.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fe0bcc700 1 -- 172.21.15.79:0/1898758491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fdc156ae0 msgr2=0x7f2fdc15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:48.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fe0bcc700 1 --2- 172.21.15.79:0/1898758491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fdc156ae0 0x7f2fdc15d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:48.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fe0bcc700 1 -- 172.21.15.79:0/1898758491 >> v2:172.21.15.155:3300/0 conn(0x7f2fdc0cc060 msgr2=0x7f2fdc0cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:48.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fe0bcc700 1 --2- 172.21.15.79:0/1898758491 >> v2:172.21.15.155:3300/0 conn(0x7f2fdc0cc060 0x7f2fdc0cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:48.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fe0bcc700 1 -- 172.21.15.79:0/1898758491 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc159950 con 0x7f2fdc071710 2023-06-23T18:36:48.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.241+0000 7f2fdb7fe700 1 --2- 172.21.15.79:0/1898758491 >> v2:172.21.15.155:3300/0 conn(0x7f2fdc0cc060 0x7f2fdc0cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:48.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.242+0000 7f2fe0bcc700 1 --2- 172.21.15.79:0/1898758491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fdc071710 0x7f2fdc071b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0d0940 tx=0x7f2fd400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa6aee8bfb29700 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:48.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.242+0000 7f2fdaffd700 1 -- 172.21.15.79:0/1898758491 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4010070 con 0x7f2fdc071710 2023-06-23T18:36:48.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.242+0000 7f2fdaffd700 1 -- 172.21.15.79:0/1898758491 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd400dd90 con 0x7f2fdc071710 2023-06-23T18:36:48.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.242+0000 7f2fdaffd700 1 -- 172.21.15.79:0/1898758491 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400b310 con 0x7f2fdc071710 2023-06-23T18:36:48.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.242+0000 7f2fe262f700 1 -- 172.21.15.79:0/1898758491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fdc071710 msgr2=0x7f2fdc071b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:48.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.242+0000 7f2fe262f700 1 --2- 172.21.15.79:0/1898758491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fdc071710 0x7f2fdc071b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0d0940 tx=0x7f2fd400d430 comp rx=0 tx=0).stop 2023-06-23T18:36:48.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.242+0000 7f2fe262f700 1 -- 172.21.15.79:0/1898758491 shutdown_connections 2023-06-23T18:36:48.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.242+0000 7f2fe262f700 1 --2- 172.21.15.79:0/1898758491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fdc156ae0 0x7f2fdc15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:48.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.242+0000 7f2fe262f700 1 --2- 172.21.15.79:0/1898758491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fdc071710 0x7f2fdc071b90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:48.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.242+0000 7f2fe262f700 1 --2- 172.21.15.79:0/1898758491 >> v2:172.21.15.155:3300/0 conn(0x7f2fdc0cc060 0x7f2fdc0cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:48.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.242+0000 7f2fe262f700 1 -- 172.21.15.79:0/1898758491 >> 172.21.15.79:0/1898758491 conn(0x7f2fdc06afa0 msgr2=0x7f2fdc06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:48.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.242+0000 7f2fe262f700 1 -- 172.21.15.79:0/1898758491 shutdown_connections 2023-06-23T18:36:48.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.242+0000 7f2fe262f700 1 -- 172.21.15.79:0/1898758491 wait complete. 2023-06-23T18:36:48.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.243+0000 7f2fe262f700 1 Processor -- start 2023-06-23T18:36:48.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.244+0000 7f2fe262f700 1 -- start start 2023-06-23T18:36:48.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.244+0000 7f2fe262f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fdc071710 0x7f2fdc1528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:48.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.244+0000 7f2fe262f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fdc0cc060 0x7f2fdc14cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:48.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.244+0000 7f2fe262f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fdc156ae0 0x7f2fdc15cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:48.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.244+0000 7f2fe262f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc14d320 con 0x7f2fdc071710 2023-06-23T18:36:48.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.244+0000 7f2fe262f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc1576b0 con 0x7f2fdc0cc060 2023-06-23T18:36:48.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.244+0000 7f2fe262f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc1f6490 con 0x7f2fdc156ae0 2023-06-23T18:36:48.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fdc071710 0x7f2fdc1528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:48.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fdc071710 0x7f2fdc1528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41216/0 (socket says 172.21.15.79:41216) 2023-06-23T18:36:48.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fdbfff700 1 -- 172.21.15.79:0/109417631 learned_addr learned my addr 172.21.15.79:0/109417631 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:48.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fdb7fe700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fdc0cc060 0x7f2fdc14cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:48.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fe0bcc700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fdc156ae0 0x7f2fdc15cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:48.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fdbfff700 1 -- 172.21.15.79:0/109417631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fdc156ae0 msgr2=0x7f2fdc15cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:48.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fdbfff700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fdc156ae0 0x7f2fdc15cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:48.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fdbfff700 1 -- 172.21.15.79:0/109417631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fdc0cc060 msgr2=0x7f2fdc14cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:48.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fdbfff700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fdc0cc060 0x7f2fdc14cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:48.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fdbfff700 1 -- 172.21.15.79:0/109417631 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd400bd40 con 0x7f2fdc071710 2023-06-23T18:36:48.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fe0bcc700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fdc156ae0 0x7f2fdc15cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:48.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fdbfff700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fdc071710 0x7f2fdc1528a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000d800 tx=0x7f2fd000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:48.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fd8ff9700 1 -- 172.21.15.79:0/109417631 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00125f0 con 0x7f2fdc071710 2023-06-23T18:36:48.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fd8ff9700 1 -- 172.21.15.79:0/109417631 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd0012750 con 0x7f2fdc071710 2023-06-23T18:36:48.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fe262f700 1 -- 172.21.15.79:0/109417631 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc157900 con 0x7f2fdc071710 2023-06-23T18:36:48.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fd8ff9700 1 -- 172.21.15.79:0/109417631 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0012b00 con 0x7f2fdc071710 2023-06-23T18:36:48.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.245+0000 7f2fe262f700 1 -- 172.21.15.79:0/109417631 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc157ec0 con 0x7f2fdc071710 2023-06-23T18:36:48.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.246+0000 7f2fe262f700 1 -- 172.21.15.79:0/109417631 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fdc0672c0 con 0x7f2fdc071710 2023-06-23T18:36:48.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.247+0000 7f2fd8ff9700 1 -- 172.21.15.79:0/109417631 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2fd0012c60 con 0x7f2fdc071710 2023-06-23T18:36:48.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.247+0000 7f2fd8ff9700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2fc806ace0 0x7f2fc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:48.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.247+0000 7f2fd8ff9700 1 -- 172.21.15.79:0/109417631 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f2fd0089830 con 0x7f2fdc071710 2023-06-23T18:36:48.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.247+0000 7f2fdb7fe700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2fc806ace0 0x7f2fc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:48.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.247+0000 7f2fdb7fe700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2fc806ace0 0x7f2fc806d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc15bfc0 tx=0x7f2fcc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:48.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.251+0000 7f2fd8ff9700 1 -- 172.21.15.79:0/109417631 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2fd0054850 con 0x7f2fdc071710 2023-06-23T18:36:48.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:48.480+0000 7f2fe262f700 1 -- 172.21.15.79:0/109417631 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2fdc0b1be0 con 0x7f2fdc071710 2023-06-23T18:36:49.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.111+0000 7f2fd8ff9700 1 -- 172.21.15.79:0/109417631 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2fd0057e70 con 0x7f2fdc071710 2023-06-23T18:36:49.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.113+0000 7f2fe262f700 1 -- 172.21.15.79:0/109417631 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2fc806ace0 msgr2=0x7f2fc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:49.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.113+0000 7f2fe262f700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2fc806ace0 0x7f2fc806d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc15bfc0 tx=0x7f2fcc008040 comp rx=0 tx=0).stop 2023-06-23T18:36:49.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.113+0000 7f2fe262f700 1 -- 172.21.15.79:0/109417631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fdc071710 msgr2=0x7f2fdc1528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:49.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.113+0000 7f2fe262f700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fdc071710 0x7f2fdc1528a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000d800 tx=0x7f2fd000dbc0 comp rx=0 tx=0).stop 2023-06-23T18:36:49.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.113+0000 7f2fe262f700 1 -- 172.21.15.79:0/109417631 shutdown_connections 2023-06-23T18:36:49.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.113+0000 7f2fe262f700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fdc156ae0 0x7f2fdc15cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:49.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.113+0000 7f2fe262f700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fdc071710 0x7f2fdc1528a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:49.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.113+0000 7f2fe262f700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2fc806ace0 0x7f2fc806d190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:49.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.113+0000 7f2fe262f700 1 --2- 172.21.15.79:0/109417631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fdc0cc060 0x7f2fdc14cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:49.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.113+0000 7f2fe262f700 1 -- 172.21.15.79:0/109417631 >> 172.21.15.79:0/109417631 conn(0x7f2fdc06afa0 msgr2=0x7f2fdc06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:49.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.114+0000 7f2fe262f700 1 -- 172.21.15.79:0/109417631 shutdown_connections 2023-06-23T18:36:49.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.114+0000 7f2fe262f700 1 -- 172.21.15.79:0/109417631 wait complete. 2023-06-23T18:36:49.133 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-06-23T18:36:49.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.299+0000 7f971eb55700 1 Processor -- start 2023-06-23T18:36:49.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.299+0000 7f971eb55700 1 -- start start 2023-06-23T18:36:49.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.300+0000 7f971eb55700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f97180b19d0 0x7f9718071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:49.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.300+0000 7f971eb55700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f971815d620 con 0x7f97180aed90 2023-06-23T18:36:49.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.300+0000 7f971eb55700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97180cd9d0 con 0x7f971815c710 2023-06-23T18:36:49.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.300+0000 7f971eb55700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97180ca870 con 0x7f97180b19d0 2023-06-23T18:36:49.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.300+0000 7f9717fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f97180b19d0 0x7f9718071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:49.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.300+0000 7f971d0f2700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f97180aed90 0x7f97180719d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44268/0 (socket says 172.21.15.79:44268) 2023-06-23T18:36:49.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.300+0000 7f971d0f2700 1 -- 172.21.15.79:0/3041431680 learned_addr learned my addr 172.21.15.79:0/3041431680 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:49.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.300+0000 7f9717fff700 1 -- 172.21.15.79:0/3041431680 >> v1:172.21.15.155:6789/0 conn(0x7f971815c710 legacy=0x7f97180b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:49.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.300+0000 7f9717fff700 1 -- 172.21.15.79:0/3041431680 >> v1:172.21.15.79:6789/0 conn(0x7f97180aed90 legacy=0x7f97180719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:49.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.300+0000 7f97177fe700 1 -- 172.21.15.79:0/3041431680 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614663042 0 0) 0x7f971815d620 con 0x7f97180aed90 2023-06-23T18:36:49.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.300+0000 7f9717fff700 1 -- 172.21.15.79:0/3041431680 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971815ccf0 con 0x7f97180b19d0 2023-06-23T18:36:49.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.300+0000 7f9717fff700 1 --2- 172.21.15.79:0/3041431680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97180b19d0 0x7f9718071490 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9704009840 tx=0x7f9704009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de4bd3130ef6520 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:49.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.300+0000 7f97177fe700 1 -- 172.21.15.79:0/3041431680 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704011830 con 0x7f97180b19d0 2023-06-23T18:36:49.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.301+0000 7f97177fe700 1 -- 172.21.15.79:0/3041431680 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9704011990 con 0x7f97180b19d0 2023-06-23T18:36:49.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.301+0000 7f97177fe700 1 -- 172.21.15.79:0/3041431680 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704011d20 con 0x7f97180b19d0 2023-06-23T18:36:49.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.301+0000 7f971eb55700 1 -- 172.21.15.79:0/3041431680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97180b19d0 msgr2=0x7f9718071490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:49.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.301+0000 7f971eb55700 1 --2- 172.21.15.79:0/3041431680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97180b19d0 0x7f9718071490 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9704009840 tx=0x7f9704009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:49.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.301+0000 7f971eb55700 1 -- 172.21.15.79:0/3041431680 shutdown_connections 2023-06-23T18:36:49.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.301+0000 7f971eb55700 1 --2- 172.21.15.79:0/3041431680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97180b19d0 0x7f9718071490 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:49.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.301+0000 7f971eb55700 1 -- 172.21.15.79:0/3041431680 >> 172.21.15.79:0/3041431680 conn(0x7f971806e200 msgr2=0x7f97180cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:49.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.301+0000 7f971eb55700 1 -- 172.21.15.79:0/3041431680 shutdown_connections 2023-06-23T18:36:49.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.301+0000 7f971eb55700 1 -- 172.21.15.79:0/3041431680 wait complete. 2023-06-23T18:36:49.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.302+0000 7f971eb55700 1 Processor -- start 2023-06-23T18:36:49.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.302+0000 7f971eb55700 1 -- start start 2023-06-23T18:36:49.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.303+0000 7f971eb55700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97180aed90 0x7f97180c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:49.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.303+0000 7f971eb55700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97180b19d0 0x7f97180c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:49.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.303+0000 7f971eb55700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f971815c710 0x7f97180b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:49.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.303+0000 7f971eb55700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97180abc60 con 0x7f971815c710 2023-06-23T18:36:49.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.303+0000 7f971eb55700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97180c83c0 con 0x7f97180aed90 2023-06-23T18:36:49.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.303+0000 7f971eb55700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97180c93b0 con 0x7f97180b19d0 2023-06-23T18:36:49.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.303+0000 7f971d0f2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f971815c710 0x7f97180b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:49.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.303+0000 7f971d0f2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f971815c710 0x7f97180b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41234/0 (socket says 172.21.15.79:41234) 2023-06-23T18:36:49.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.303+0000 7f971d0f2700 1 -- 172.21.15.79:0/2649490668 learned_addr learned my addr 172.21.15.79:0/2649490668 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:49.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.303+0000 7f971c8f1700 1 --2- 172.21.15.79:0/2649490668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97180aed90 0x7f97180c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:49.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.303+0000 7f9717fff700 1 --2- 172.21.15.79:0/2649490668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97180b19d0 0x7f97180c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:49.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.304+0000 7f971d0f2700 1 -- 172.21.15.79:0/2649490668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97180b19d0 msgr2=0x7f97180c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:49.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.304+0000 7f971d0f2700 1 --2- 172.21.15.79:0/2649490668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97180b19d0 0x7f97180c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:49.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.304+0000 7f971d0f2700 1 -- 172.21.15.79:0/2649490668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97180aed90 msgr2=0x7f97180c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:49.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.304+0000 7f971d0f2700 1 --2- 172.21.15.79:0/2649490668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97180aed90 0x7f97180c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:49.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.304+0000 7f971d0f2700 1 -- 172.21.15.79:0/2649490668 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97040094a0 con 0x7f971815c710 2023-06-23T18:36:49.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.304+0000 7f971d0f2700 1 --2- 172.21.15.79:0/2649490668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f971815c710 0x7f97180b48f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9710002230 tx=0x7f971000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:49.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.304+0000 7f97157fa700 1 -- 172.21.15.79:0/2649490668 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971000aac0 con 0x7f971815c710 2023-06-23T18:36:49.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.304+0000 7f97157fa700 1 -- 172.21.15.79:0/2649490668 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971000ac40 con 0x7f971815c710 2023-06-23T18:36:49.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.304+0000 7f971eb55700 1 -- 172.21.15.79:0/2649490668 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97180c7a90 con 0x7f971815c710 2023-06-23T18:36:49.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.304+0000 7f97157fa700 1 -- 172.21.15.79:0/2649490668 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97100070d0 con 0x7f971815c710 2023-06-23T18:36:49.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.304+0000 7f971eb55700 1 -- 172.21.15.79:0/2649490668 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97180c8050 con 0x7f971815c710 2023-06-23T18:36:49.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.305+0000 7f971eb55700 1 -- 172.21.15.79:0/2649490668 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971806bf30 con 0x7f971815c710 2023-06-23T18:36:49.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.305+0000 7f97157fa700 1 -- 172.21.15.79:0/2649490668 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9710007250 con 0x7f971815c710 2023-06-23T18:36:49.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.306+0000 7f97157fa700 1 --2- 172.21.15.79:0/2649490668 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f96fc06ace0 0x7f96fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:49.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.306+0000 7f97157fa700 1 -- 172.21.15.79:0/2649490668 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f97100892a0 con 0x7f971815c710 2023-06-23T18:36:49.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.306+0000 7f971c8f1700 1 --2- 172.21.15.79:0/2649490668 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f96fc06ace0 0x7f96fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:49.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.306+0000 7f971c8f1700 1 --2- 172.21.15.79:0/2649490668 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f96fc06ace0 0x7f96fc06d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f970c007fd0 tx=0x7f970c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:49.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.310+0000 7f97157fa700 1 -- 172.21.15.79:0/2649490668 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f97100542c0 con 0x7f971815c710 2023-06-23T18:36:49.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:49.497+0000 7f971eb55700 1 -- 172.21.15.79:0/2649490668 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f97180cbc20 con 0x7f971815c710 2023-06-23T18:36:50.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.112+0000 7f97157fa700 1 -- 172.21.15.79:0/2649490668 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f97100578e0 con 0x7f971815c710 2023-06-23T18:36:50.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.115+0000 7f971eb55700 1 -- 172.21.15.79:0/2649490668 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f96fc06ace0 msgr2=0x7f96fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:50.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.115+0000 7f971eb55700 1 --2- 172.21.15.79:0/2649490668 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f96fc06ace0 0x7f96fc06d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f970c007fd0 tx=0x7f970c009040 comp rx=0 tx=0).stop 2023-06-23T18:36:50.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.115+0000 7f971eb55700 1 -- 172.21.15.79:0/2649490668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f971815c710 msgr2=0x7f97180b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:50.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.115+0000 7f971eb55700 1 --2- 172.21.15.79:0/2649490668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f971815c710 0x7f97180b48f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9710002230 tx=0x7f971000cf50 comp rx=0 tx=0).stop 2023-06-23T18:36:50.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.115+0000 7f971eb55700 1 -- 172.21.15.79:0/2649490668 shutdown_connections 2023-06-23T18:36:50.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.115+0000 7f971eb55700 1 --2- 172.21.15.79:0/2649490668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f971815c710 0x7f97180b48f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:50.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.115+0000 7f971eb55700 1 --2- 172.21.15.79:0/2649490668 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f96fc06ace0 0x7f96fc06d190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:50.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.115+0000 7f971eb55700 1 --2- 172.21.15.79:0/2649490668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97180aed90 0x7f97180c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:50.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.115+0000 7f971eb55700 1 --2- 172.21.15.79:0/2649490668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97180b19d0 0x7f97180c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:50.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.115+0000 7f971eb55700 1 -- 172.21.15.79:0/2649490668 >> 172.21.15.79:0/2649490668 conn(0x7f971806e200 msgr2=0x7f97180af990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:50.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.115+0000 7f971eb55700 1 -- 172.21.15.79:0/2649490668 shutdown_connections 2023-06-23T18:36:50.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.115+0000 7f971eb55700 1 -- 172.21.15.79:0/2649490668 wait complete. 2023-06-23T18:36:50.122 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_metadata' removed 2023-06-23T18:36:50.133 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-06-23T18:36:50.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.253+0000 7f10488fa700 1 Processor -- start 2023-06-23T18:36:50.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.253+0000 7f10488fa700 1 -- start start 2023-06-23T18:36:50.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.254+0000 7f10488fa700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f104415c710 0x7f10440b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:50.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.254+0000 7f10488fa700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f10440aed90 0x7f10440b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:50.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.254+0000 7f10488fa700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104415d620 con 0x7f10440cd410 2023-06-23T18:36:50.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.254+0000 7f10488fa700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10440ca870 con 0x7f10440aed90 2023-06-23T18:36:50.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.254+0000 7f10488fa700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10440724a0 con 0x7f104415c710 2023-06-23T18:36:50.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.254+0000 7f1041d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f10440aed90 0x7f10440b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.254+0000 7f1042d9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f10440cd410 0x7f104406eeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44280/0 (socket says 172.21.15.79:44280) 2023-06-23T18:36:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.254+0000 7f1042d9d700 1 -- 172.21.15.79:0/2710076166 learned_addr learned my addr 172.21.15.79:0/2710076166 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.254+0000 7f1041d9b700 1 --2- 172.21.15.79:0/2710076166 >> v2:172.21.15.79:3300/0 conn(0x7f10440aed90 0x7f10440b1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41238/0 (socket says 172.21.15.79:41238) 2023-06-23T18:36:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.255+0000 7f104259c700 1 --2- 172.21.15.79:0/2710076166 >> v2:172.21.15.155:3300/0 conn(0x7f104415c710 0x7f10440b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.255+0000 7f1041d9b700 1 -- 172.21.15.79:0/2710076166 >> v2:172.21.15.155:3300/0 conn(0x7f104415c710 msgr2=0x7f10440b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.255+0000 7f1041d9b700 1 --2- 172.21.15.79:0/2710076166 >> v2:172.21.15.155:3300/0 conn(0x7f104415c710 0x7f10440b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.255+0000 7f1041d9b700 1 -- 172.21.15.79:0/2710076166 >> v1:172.21.15.79:6789/0 conn(0x7f10440cd410 legacy=0x7f104406eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.255+0000 7f1041d9b700 1 -- 172.21.15.79:0/2710076166 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10440cd9d0 con 0x7f10440aed90 2023-06-23T18:36:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.255+0000 7f104159a700 1 -- 172.21.15.79:0/2710076166 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886775857 0 0) 0x7f104415d620 con 0x7f10440cd410 2023-06-23T18:36:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.255+0000 7f104259c700 1 --2- 172.21.15.79:0/2710076166 >> v2:172.21.15.155:3300/0 conn(0x7f104415c710 0x7f10440b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.255+0000 7f1041d9b700 1 --2- 172.21.15.79:0/2710076166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10440aed90 0x7f10440b1800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f10440d0910 tx=0x7f1034009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03216288fd4e9b9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.255+0000 7f104159a700 1 -- 172.21.15.79:0/2710076166 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034011830 con 0x7f10440aed90 2023-06-23T18:36:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.255+0000 7f104159a700 1 -- 172.21.15.79:0/2710076166 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1034011990 con 0x7f10440aed90 2023-06-23T18:36:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.255+0000 7f104159a700 1 -- 172.21.15.79:0/2710076166 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034011d20 con 0x7f10440aed90 2023-06-23T18:36:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.256+0000 7f10488fa700 1 -- 172.21.15.79:0/2710076166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10440aed90 msgr2=0x7f10440b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:50.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.256+0000 7f10488fa700 1 --2- 172.21.15.79:0/2710076166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10440aed90 0x7f10440b1800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f10440d0910 tx=0x7f1034009b20 comp rx=0 tx=0).stop 2023-06-23T18:36:50.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.256+0000 7f10488fa700 1 -- 172.21.15.79:0/2710076166 shutdown_connections 2023-06-23T18:36:50.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.256+0000 7f10488fa700 1 --2- 172.21.15.79:0/2710076166 >> v2:172.21.15.155:3300/0 conn(0x7f104415c710 0x7f10440b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:50.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.256+0000 7f10488fa700 1 --2- 172.21.15.79:0/2710076166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10440aed90 0x7f10440b1800 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:50.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.256+0000 7f10488fa700 1 -- 172.21.15.79:0/2710076166 >> 172.21.15.79:0/2710076166 conn(0x7f104406e200 msgr2=0x7f10440cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:50.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.256+0000 7f10488fa700 1 -- 172.21.15.79:0/2710076166 shutdown_connections 2023-06-23T18:36:50.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.256+0000 7f10488fa700 1 -- 172.21.15.79:0/2710076166 wait complete. 2023-06-23T18:36:50.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.257+0000 7f10488fa700 1 Processor -- start 2023-06-23T18:36:50.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.257+0000 7f10488fa700 1 -- start start 2023-06-23T18:36:50.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f10488fa700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10440aed90 0x7f10440c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:50.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f10488fa700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10440cd410 0x7f10440ac570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:50.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f10488fa700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f104415c710 0x7f10440acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:50.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f10488fa700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10441518a0 con 0x7f10440aed90 2023-06-23T18:36:50.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f10488fa700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1044150f90 con 0x7f104415c710 2023-06-23T18:36:50.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f10488fa700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10440ad340 con 0x7f10440cd410 2023-06-23T18:36:50.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f104259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10440aed90 0x7f10440c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:50.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f104259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10440aed90 0x7f10440c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41244/0 (socket says 172.21.15.79:41244) 2023-06-23T18:36:50.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f104259c700 1 -- 172.21.15.79:0/1204929595 learned_addr learned my addr 172.21.15.79:0/1204929595 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:50.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f104259c700 1 -- 172.21.15.79:0/1204929595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10440cd410 msgr2=0x7f10440ac570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:50.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f104259c700 1 --2- 172.21.15.79:0/1204929595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10440cd410 0x7f10440ac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:50.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f104259c700 1 -- 172.21.15.79:0/1204929595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f104415c710 msgr2=0x7f10440acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:50.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f104259c700 1 --2- 172.21.15.79:0/1204929595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f104415c710 0x7f10440acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:50.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f104259c700 1 -- 172.21.15.79:0/1204929595 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10340094a0 con 0x7f10440aed90 2023-06-23T18:36:50.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.258+0000 7f104259c700 1 --2- 172.21.15.79:0/1204929595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10440aed90 0x7f10440c55d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f103800e810 tx=0x7f103800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:50.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.259+0000 7f1032ffd700 1 -- 172.21.15.79:0/1204929595 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1038004230 con 0x7f10440aed90 2023-06-23T18:36:50.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.259+0000 7f1032ffd700 1 -- 172.21.15.79:0/1204929595 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1038004390 con 0x7f10440aed90 2023-06-23T18:36:50.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.259+0000 7f10488fa700 1 -- 172.21.15.79:0/1204929595 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10441511e0 con 0x7f10440aed90 2023-06-23T18:36:50.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.259+0000 7f1032ffd700 1 -- 172.21.15.79:0/1204929595 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10380036a0 con 0x7f10440aed90 2023-06-23T18:36:50.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.259+0000 7f10488fa700 1 -- 172.21.15.79:0/1204929595 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10440acff0 con 0x7f10440aed90 2023-06-23T18:36:50.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.260+0000 7f10488fa700 1 -- 172.21.15.79:0/1204929595 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104406bf30 con 0x7f10440aed90 2023-06-23T18:36:50.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.260+0000 7f1032ffd700 1 -- 172.21.15.79:0/1204929595 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1038003800 con 0x7f10440aed90 2023-06-23T18:36:50.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.260+0000 7f1032ffd700 1 --2- 172.21.15.79:0/1204929595 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f102406ace0 0x7f102406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:50.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.261+0000 7f1032ffd700 1 -- 172.21.15.79:0/1204929595 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f1038088e20 con 0x7f10440aed90 2023-06-23T18:36:50.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.261+0000 7f1041d9b700 1 --2- 172.21.15.79:0/1204929595 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f102406ace0 0x7f102406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:50.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.261+0000 7f1041d9b700 1 --2- 172.21.15.79:0/1204929595 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f102406ace0 0x7f102406d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f10441f59b0 tx=0x7f1034010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:50.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.265+0000 7f1032ffd700 1 -- 172.21.15.79:0/1204929595 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1038054080 con 0x7f10440aed90 2023-06-23T18:36:50.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:50.454+0000 7f10488fa700 1 -- 172.21.15.79:0/1204929595 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f10440c9e30 con 0x7f10440aed90 2023-06-23T18:36:51.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.126+0000 7f1032ffd700 1 -- 172.21.15.79:0/1204929595 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f10380576a0 con 0x7f10440aed90 2023-06-23T18:36:51.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.129+0000 7f10488fa700 1 -- 172.21.15.79:0/1204929595 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f102406ace0 msgr2=0x7f102406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.129+0000 7f10488fa700 1 --2- 172.21.15.79:0/1204929595 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f102406ace0 0x7f102406d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f10441f59b0 tx=0x7f1034010000 comp rx=0 tx=0).stop 2023-06-23T18:36:51.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.129+0000 7f10488fa700 1 -- 172.21.15.79:0/1204929595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10440aed90 msgr2=0x7f10440c55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.129+0000 7f10488fa700 1 --2- 172.21.15.79:0/1204929595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10440aed90 0x7f10440c55d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f103800e810 tx=0x7f103800ebd0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.129+0000 7f10488fa700 1 -- 172.21.15.79:0/1204929595 shutdown_connections 2023-06-23T18:36:51.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.129+0000 7f10488fa700 1 --2- 172.21.15.79:0/1204929595 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f102406ace0 0x7f102406d190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.129+0000 7f10488fa700 1 --2- 172.21.15.79:0/1204929595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f104415c710 0x7f10440acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.129+0000 7f10488fa700 1 --2- 172.21.15.79:0/1204929595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10440aed90 0x7f10440c55d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.129+0000 7f10488fa700 1 --2- 172.21.15.79:0/1204929595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10440cd410 0x7f10440ac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.129+0000 7f10488fa700 1 -- 172.21.15.79:0/1204929595 >> 172.21.15.79:0/1204929595 conn(0x7f104406e200 msgr2=0x7f10440af310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:51.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.129+0000 7f10488fa700 1 -- 172.21.15.79:0/1204929595 shutdown_connections 2023-06-23T18:36:51.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.129+0000 7f10488fa700 1 -- 172.21.15.79:0/1204929595 wait complete. 2023-06-23T18:36:51.136 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data' removed 2023-06-23T18:36:51.151 INFO:tasks.ceph.mds.a:Restarting daemon 2023-06-23T18:36:51.152 INFO:tasks.ceph.mds.a:Stopping old one... 2023-06-23T18:36:51.152 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-06-23T18:36:51.152 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:36:51.153 INFO:tasks.ceph.mds.c:Restarting daemon 2023-06-23T18:36:51.153 INFO:tasks.ceph.mds.c:Stopping old one... 2023-06-23T18:36:51.153 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-06-23T18:36:51.153 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:36:51.154 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-23T18:36:51.154 INFO:tasks.ceph.mds.b:Stopping old one... 2023-06-23T18:36:51.154 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-23T18:36:51.154 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:36:51.154 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-23T18:36:51.155 INFO:tasks.ceph.mds.d:Stopping old one... 2023-06-23T18:36:51.155 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-23T18:36:51.155 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:36:51.163 INFO:tasks.ceph.mds.c:Stopped 2023-06-23T18:36:51.164 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-06-23T18:36:51.165 INFO:tasks.ceph.mds.a:Stopped 2023-06-23T18:36:51.165 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-06-23T18:36:51.166 INFO:tasks.ceph.mds.b:Stopped 2023-06-23T18:36:51.166 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-23T18:36:51.167 INFO:tasks.ceph.mds.d:Stopped 2023-06-23T18:36:51.167 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-23T18:36:51.206 INFO:tasks.ceph.mds.c:Started 2023-06-23T18:36:51.206 DEBUG:teuthology.parallel:result is None 2023-06-23T18:36:51.207 INFO:tasks.ceph.mds.a:Started 2023-06-23T18:36:51.208 DEBUG:teuthology.parallel:result is None 2023-06-23T18:36:51.209 INFO:tasks.ceph.mds.b:Started 2023-06-23T18:36:51.209 DEBUG:teuthology.parallel:result is None 2023-06-23T18:36:51.210 INFO:tasks.ceph.mds.d:Started 2023-06-23T18:36:51.210 DEBUG:teuthology.parallel:result is None 2023-06-23T18:36:51.210 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-06-23T18:36:51.295 INFO:tasks.ceph.mds.c.smithi079.stdout:starting mds.c at 2023-06-23T18:36:51.296 INFO:tasks.ceph.mds.a.smithi079.stdout:starting mds.a at 2023-06-23T18:36:51.298 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-06-23T18:36:51.298 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-06-23T18:36:51.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.349+0000 7f34f5658700 1 Processor -- start 2023-06-23T18:36:51.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.349+0000 7f34f5658700 1 -- start start 2023-06-23T18:36:51.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.350+0000 7f34f5658700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f34f00b83c0 0x7f34f00cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.350+0000 7f34f5658700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f34f00cfc60 0x7f34f00d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.350+0000 7f34f5658700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f00d0a80 con 0x7f34f00d3710 2023-06-23T18:36:51.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.350+0000 7f34f5658700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f00b8790 con 0x7f34f00cfc60 2023-06-23T18:36:51.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.350+0000 7f34f5658700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f00b4e70 con 0x7f34f00b83c0 2023-06-23T18:36:51.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.350+0000 7f34ef7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f34f00cfc60 0x7f34f00d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.350+0000 7f34ef7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f34f00cfc60 0x7f34f00d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41278/0 (socket says 172.21.15.79:41278) 2023-06-23T18:36:51.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.350+0000 7f34ef7fe700 1 -- 172.21.15.79:0/3553461395 learned_addr learned my addr 172.21.15.79:0/3553461395 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:51.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.350+0000 7f34eeffd700 1 --2- 172.21.15.79:0/3553461395 >> v2:172.21.15.155:3300/0 conn(0x7f34f00b83c0 0x7f34f00cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.351+0000 7f34ef7fe700 1 -- 172.21.15.79:0/3553461395 >> v2:172.21.15.155:3300/0 conn(0x7f34f00b83c0 msgr2=0x7f34f00cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:51.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.351+0000 7f34ef7fe700 1 --2- 172.21.15.79:0/3553461395 >> v2:172.21.15.155:3300/0 conn(0x7f34f00b83c0 0x7f34f00cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.351+0000 7f34ef7fe700 1 -- 172.21.15.79:0/3553461395 >> v1:172.21.15.155:6789/0 conn(0x7f34f00d3710 legacy=0x7f34f00cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.351+0000 7f34ef7fe700 1 -- 172.21.15.79:0/3553461395 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f00b5150 con 0x7f34f00cfc60 2023-06-23T18:36:51.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.351+0000 7f34ef7fe700 1 --2- 172.21.15.79:0/3553461395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34f00cfc60 0x7f34f00d0f10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f34f00cf520 tx=0x7f34e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35daa48c736830a7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:51.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.351+0000 7f34edffb700 1 -- 172.21.15.79:0/3553461395 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e800e070 con 0x7f34f00cfc60 2023-06-23T18:36:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.351+0000 7f34edffb700 1 -- 172.21.15.79:0/3553461395 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e800b910 con 0x7f34f00cfc60 2023-06-23T18:36:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.351+0000 7f34edffb700 1 -- 172.21.15.79:0/3553461395 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e800bcc0 con 0x7f34f00cfc60 2023-06-23T18:36:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.351+0000 7f34f5658700 1 -- 172.21.15.79:0/3553461395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34f00cfc60 msgr2=0x7f34f00d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.351+0000 7f34f5658700 1 --2- 172.21.15.79:0/3553461395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34f00cfc60 0x7f34f00d0f10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f34f00cf520 tx=0x7f34e800b330 comp rx=0 tx=0).stop 2023-06-23T18:36:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.351+0000 7f34f5658700 1 -- 172.21.15.79:0/3553461395 shutdown_connections 2023-06-23T18:36:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.351+0000 7f34f5658700 1 --2- 172.21.15.79:0/3553461395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34f00cfc60 0x7f34f00d0f10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.351+0000 7f34f5658700 1 --2- 172.21.15.79:0/3553461395 >> v2:172.21.15.155:3300/0 conn(0x7f34f00b83c0 0x7f34f00cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.351+0000 7f34f5658700 1 -- 172.21.15.79:0/3553461395 >> 172.21.15.79:0/3553461395 conn(0x7f34f006b350 msgr2=0x7f34f006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:51.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.352+0000 7f34f5658700 1 -- 172.21.15.79:0/3553461395 shutdown_connections 2023-06-23T18:36:51.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.352+0000 7f34f5658700 1 -- 172.21.15.79:0/3553461395 wait complete. 2023-06-23T18:36:51.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.353+0000 7f34f5658700 1 Processor -- start 2023-06-23T18:36:51.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.353+0000 7f34f5658700 1 -- start start 2023-06-23T18:36:51.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34f5658700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34f00b83c0 0x7f34f00c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34f5658700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34f00cfc60 0x7f34f014cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34f5658700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34f00d3710 0x7f34f015e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34f5658700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f014d510 con 0x7f34f00cfc60 2023-06-23T18:36:51.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34f5658700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00cb000 con 0x7f34f00b83c0 2023-06-23T18:36:51.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34f5658700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00cb2e0 con 0x7f34f00d3710 2023-06-23T18:36:51.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34ee7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34f00cfc60 0x7f34f014cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34ee7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34f00cfc60 0x7f34f014cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41282/0 (socket says 172.21.15.79:41282) 2023-06-23T18:36:51.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34ee7fc700 1 -- 172.21.15.79:0/401369158 learned_addr learned my addr 172.21.15.79:0/401369158 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:51.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34eeffd700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34f00b83c0 0x7f34f00c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34ef7fe700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34f00d3710 0x7f34f015e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34ee7fc700 1 -- 172.21.15.79:0/401369158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34f00d3710 msgr2=0x7f34f015e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34ee7fc700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34f00d3710 0x7f34f015e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34ee7fc700 1 -- 172.21.15.79:0/401369158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34f00b83c0 msgr2=0x7f34f00c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34ee7fc700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34f00b83c0 0x7f34f00c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34ee7fc700 1 -- 172.21.15.79:0/401369158 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e8009b30 con 0x7f34f00cfc60 2023-06-23T18:36:51.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34ef7fe700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34f00d3710 0x7f34f015e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:51.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34ee7fc700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34f00cfc60 0x7f34f014cfd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f34e400c4b0 tx=0x7f34e400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:51.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34d77fe700 1 -- 172.21.15.79:0/401369158 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e40052d0 con 0x7f34f00cfc60 2023-06-23T18:36:51.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34d77fe700 1 -- 172.21.15.79:0/401369158 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e4005430 con 0x7f34f00cfc60 2023-06-23T18:36:51.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34f5658700 1 -- 172.21.15.79:0/401369158 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f00cb5c0 con 0x7f34f00cfc60 2023-06-23T18:36:51.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34d77fe700 1 -- 172.21.15.79:0/401369158 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e40038c0 con 0x7f34f00cfc60 2023-06-23T18:36:51.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.354+0000 7f34f5658700 1 -- 172.21.15.79:0/401369158 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f00cbb80 con 0x7f34f00cfc60 2023-06-23T18:36:51.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.355+0000 7f34f5658700 1 -- 172.21.15.79:0/401369158 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f00cbd40 con 0x7f34f00cfc60 2023-06-23T18:36:51.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.356+0000 7f34d77fe700 1 -- 172.21.15.79:0/401369158 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f34e4003a20 con 0x7f34f00cfc60 2023-06-23T18:36:51.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.356+0000 7f34d77fe700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f34dc06ace0 0x7f34dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.356+0000 7f34eeffd700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f34dc06ace0 0x7f34dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.356+0000 7f34d77fe700 1 -- 172.21.15.79:0/401369158 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f34e40884b0 con 0x7f34f00cfc60 2023-06-23T18:36:51.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.357+0000 7f34eeffd700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f34dc06ace0 0x7f34dc06d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009b40 tx=0x7f34e0009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:51.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.361+0000 7f34d77fe700 1 -- 172.21.15.79:0/401369158 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f34e40538f0 con 0x7f34f00cfc60 2023-06-23T18:36:51.572 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:51.568+0000 7f34f5658700 1 -- 172.21.15.79:0/401369158 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f34f00d1ca0 con 0x7f34f00cfc60 2023-06-23T18:36:52.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.327+0000 7f34d77fe700 1 -- 172.21.15.79:0/401369158 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f34e4056f10 con 0x7f34f00cfc60 2023-06-23T18:36:52.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.329+0000 7f34f5658700 1 -- 172.21.15.79:0/401369158 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f34dc06ace0 msgr2=0x7f34dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:52.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.329+0000 7f34f5658700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f34dc06ace0 0x7f34dc06d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009b40 tx=0x7f34e0009210 comp rx=0 tx=0).stop 2023-06-23T18:36:52.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.329+0000 7f34f5658700 1 -- 172.21.15.79:0/401369158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34f00cfc60 msgr2=0x7f34f014cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:52.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.329+0000 7f34f5658700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34f00cfc60 0x7f34f014cfd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f34e400c4b0 tx=0x7f34e400c7c0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.330+0000 7f34f5658700 1 -- 172.21.15.79:0/401369158 shutdown_connections 2023-06-23T18:36:52.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.330+0000 7f34f5658700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34f00d3710 0x7f34f015e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.330+0000 7f34f5658700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f34dc06ace0 0x7f34dc06d190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.330+0000 7f34f5658700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34f00b83c0 0x7f34f00c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.330+0000 7f34f5658700 1 --2- 172.21.15.79:0/401369158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34f00cfc60 0x7f34f014cfd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.330+0000 7f34f5658700 1 -- 172.21.15.79:0/401369158 >> 172.21.15.79:0/401369158 conn(0x7f34f006b350 msgr2=0x7f34f00b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:52.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.330+0000 7f34f5658700 1 -- 172.21.15.79:0/401369158 shutdown_connections 2023-06-23T18:36:52.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.330+0000 7f34f5658700 1 -- 172.21.15.79:0/401369158 wait complete. 2023-06-23T18:36:52.337 INFO:teuthology.orchestra.run.smithi079.stderr: removed all blocklist entries 2023-06-23T18:36:52.348 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-06-23T18:36:52.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.468+0000 7f331e926700 1 Processor -- start 2023-06-23T18:36:52.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.468+0000 7f331e926700 1 -- start start 2023-06-23T18:36:52.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.469+0000 7f331e926700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3318071490 0x7f3318071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:52.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.469+0000 7f331e926700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33180aee30 0x7f3318071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:52.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.469+0000 7f331e926700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33181608f0 con 0x7f33180b1270 2023-06-23T18:36:52.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.469+0000 7f331e926700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33180722e0 con 0x7f3318071490 2023-06-23T18:36:52.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.469+0000 7f331e926700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3318160e20 con 0x7f33180aee30 2023-06-23T18:36:52.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.469+0000 7f3317fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3318071490 0x7f3318071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:52.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.469+0000 7f33177fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f33180b1270 0x7f3318069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44334/0 (socket says 172.21.15.79:44334) 2023-06-23T18:36:52.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.469+0000 7f33177fe700 1 -- 172.21.15.79:0/1756355909 learned_addr learned my addr 172.21.15.79:0/1756355909 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:52.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.469+0000 7f3317fff700 1 -- 172.21.15.79:0/1756355909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33180aee30 msgr2=0x7f3318071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:52.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.469+0000 7f331cec3700 1 --2- 172.21.15.79:0/1756355909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33180aee30 0x7f3318071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:52.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.469+0000 7f3316ffd700 1 -- 172.21.15.79:0/1756355909 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296524354 0 0) 0x7f33181608f0 con 0x7f33180b1270 2023-06-23T18:36:52.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.469+0000 7f3317fff700 1 --2- 172.21.15.79:0/1756355909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33180aee30 0x7f3318071da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.470+0000 7f3317fff700 1 -- 172.21.15.79:0/1756355909 >> v1:172.21.15.79:6789/0 conn(0x7f33180b1270 legacy=0x7f3318069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:52.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.470+0000 7f3317fff700 1 -- 172.21.15.79:0/1756355909 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33180cd4f0 con 0x7f3318071490 2023-06-23T18:36:52.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.470+0000 7f3317fff700 1 --2- 172.21.15.79:0/1756355909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3318071490 0x7f3318071860 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f330c009840 tx=0x7f330c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe53099b821fa81 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:52.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.470+0000 7f3316ffd700 1 -- 172.21.15.79:0/1756355909 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c011830 con 0x7f3318071490 2023-06-23T18:36:52.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.470+0000 7f3316ffd700 1 -- 172.21.15.79:0/1756355909 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330c011990 con 0x7f3318071490 2023-06-23T18:36:52.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.470+0000 7f3316ffd700 1 -- 172.21.15.79:0/1756355909 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c011d20 con 0x7f3318071490 2023-06-23T18:36:52.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.470+0000 7f331e926700 1 -- 172.21.15.79:0/1756355909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3318071490 msgr2=0x7f3318071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:52.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.470+0000 7f331e926700 1 --2- 172.21.15.79:0/1756355909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3318071490 0x7f3318071860 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f330c009840 tx=0x7f330c009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:52.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.470+0000 7f331e926700 1 -- 172.21.15.79:0/1756355909 shutdown_connections 2023-06-23T18:36:52.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.470+0000 7f331e926700 1 --2- 172.21.15.79:0/1756355909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33180aee30 0x7f3318071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.470+0000 7f331e926700 1 --2- 172.21.15.79:0/1756355909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3318071490 0x7f3318071860 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.470+0000 7f331e926700 1 -- 172.21.15.79:0/1756355909 >> 172.21.15.79:0/1756355909 conn(0x7f331806d660 msgr2=0x7f331806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:52.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.470+0000 7f331e926700 1 -- 172.21.15.79:0/1756355909 shutdown_connections 2023-06-23T18:36:52.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.470+0000 7f331e926700 1 -- 172.21.15.79:0/1756355909 wait complete. 2023-06-23T18:36:52.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.472+0000 7f331e926700 1 Processor -- start 2023-06-23T18:36:52.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.472+0000 7f331e926700 1 -- start start 2023-06-23T18:36:52.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.472+0000 7f331e926700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3318071490 0x7f33180c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:52.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.472+0000 7f331e926700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f33180aee30 0x7f33180cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:52.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.472+0000 7f331e926700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33180b1270 0x7f3318152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:52.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.472+0000 7f331e926700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33180c9820 con 0x7f33180aee30 2023-06-23T18:36:52.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.472+0000 7f331e926700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33180c6d50 con 0x7f33180b1270 2023-06-23T18:36:52.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.472+0000 7f331e926700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33180c8ef0 con 0x7f3318071490 2023-06-23T18:36:52.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f33177fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f33180aee30 0x7f33180cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:52.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f33177fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f33180aee30 0x7f33180cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41298/0 (socket says 172.21.15.79:41298) 2023-06-23T18:36:52.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f33177fe700 1 -- 172.21.15.79:0/2046642626 learned_addr learned my addr 172.21.15.79:0/2046642626 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:52.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f3317fff700 1 --2- 172.21.15.79:0/2046642626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3318071490 0x7f33180c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:52.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f331cec3700 1 --2- 172.21.15.79:0/2046642626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33180b1270 0x7f3318152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:52.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f33177fe700 1 -- 172.21.15.79:0/2046642626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3318071490 msgr2=0x7f33180c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:52.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f33177fe700 1 --2- 172.21.15.79:0/2046642626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3318071490 0x7f33180c5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f33177fe700 1 -- 172.21.15.79:0/2046642626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33180b1270 msgr2=0x7f3318152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:52.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f33177fe700 1 --2- 172.21.15.79:0/2046642626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33180b1270 0x7f3318152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f33177fe700 1 -- 172.21.15.79:0/2046642626 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c0094a0 con 0x7f33180aee30 2023-06-23T18:36:52.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f33177fe700 1 --2- 172.21.15.79:0/2046642626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f33180aee30 0x7f33180cf850 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f330800c460 tx=0x7f330800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:52.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f3314ff9700 1 -- 172.21.15.79:0/2046642626 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308004c80 con 0x7f33180aee30 2023-06-23T18:36:52.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f3314ff9700 1 -- 172.21.15.79:0/2046642626 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3308004e00 con 0x7f33180aee30 2023-06-23T18:36:52.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f331e926700 1 -- 172.21.15.79:0/2046642626 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33180c91d0 con 0x7f33180aee30 2023-06-23T18:36:52.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f3314ff9700 1 -- 172.21.15.79:0/2046642626 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308007310 con 0x7f33180aee30 2023-06-23T18:36:52.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.473+0000 7f331e926700 1 -- 172.21.15.79:0/2046642626 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3318154190 con 0x7f33180aee30 2023-06-23T18:36:52.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.474+0000 7f331e926700 1 -- 172.21.15.79:0/2046642626 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33180724a0 con 0x7f33180aee30 2023-06-23T18:36:52.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.474+0000 7f3314ff9700 1 -- 172.21.15.79:0/2046642626 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3308007470 con 0x7f33180aee30 2023-06-23T18:36:52.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.475+0000 7f3314ff9700 1 --2- 172.21.15.79:0/2046642626 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f330406ace0 0x7f330406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:52.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.475+0000 7f3314ff9700 1 -- 172.21.15.79:0/2046642626 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3308088e70 con 0x7f33180aee30 2023-06-23T18:36:52.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.475+0000 7f3317fff700 1 --2- 172.21.15.79:0/2046642626 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f330406ace0 0x7f330406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:52.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.475+0000 7f3317fff700 1 --2- 172.21.15.79:0/2046642626 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f330406ace0 0x7f330406d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f330c000c00 tx=0x7f330c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:52.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.479+0000 7f3314ff9700 1 -- 172.21.15.79:0/2046642626 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f33080541e0 con 0x7f33180aee30 2023-06-23T18:36:52.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.742+0000 7f331e926700 1 -- 172.21.15.79:0/2046642626 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f33180b5770 con 0x7f33180aee30 2023-06-23T18:36:52.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.743+0000 7f3314ff9700 1 -- 172.21.15.79:0/2046642626 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f3308057800 con 0x7f33180aee30 2023-06-23T18:36:52.748 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:52.749 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-23T18:36:52.749 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_dump": [ 2023-06-23T18:36:52.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.749 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mds.a", 2023-06-23T18:36:52.750 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDV5ZVkRuZGFRAAHZmzHrtnBMKcH3vZc5GTMQ==", 2023-06-23T18:36:52.750 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.750 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-23T18:36:52.750 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2023-06-23T18:36:52.750 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2023-06-23T18:36:52.750 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.751 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mds.b", 2023-06-23T18:36:52.751 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDV5ZVkKDaOIRAAU6cy09C9y6xYdKckBmHgXg==", 2023-06-23T18:36:52.751 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.751 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-23T18:36:52.751 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2023-06-23T18:36:52.751 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2023-06-23T18:36:52.752 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.752 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mds.c", 2023-06-23T18:36:52.752 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDV5ZVk+s6HHBAAs45yP9HZPsIUQ2b5iotFOQ==", 2023-06-23T18:36:52.752 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.753 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-23T18:36:52.753 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2023-06-23T18:36:52.753 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2023-06-23T18:36:52.753 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.753 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.754 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mds.d", 2023-06-23T18:36:52.754 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDV5ZVkJcPYKBAAHsrS6wdO7Bo+wytNDcbHgA==", 2023-06-23T18:36:52.754 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.754 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-23T18:36:52.754 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2023-06-23T18:36:52.754 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2023-06-23T18:36:52.754 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.755 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.755 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.0", 2023-06-23T18:36:52.755 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDZ5ZVkGIZ8DRAANJGztf5I1J0TeXhbtfaR6g==", 2023-06-23T18:36:52.755 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.755 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-23T18:36:52.756 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-23T18:36:52.756 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.756 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.756 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.1", 2023-06-23T18:36:52.756 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDc5ZVkNCyLDRAA3Nl41mzAhsM9zN4ZXoWavw==", 2023-06-23T18:36:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-23T18:36:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-23T18:36:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.2", 2023-06-23T18:36:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDf5ZVkRBjlDBAA2jC3fNJysHoW/tMJGmyv9Q==", 2023-06-23T18:36:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-23T18:36:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-23T18:36:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.3", 2023-06-23T18:36:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDi5ZVkisr5CxAArmGjhDtCQiKzqwQw29/L1w==", 2023-06-23T18:36:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-23T18:36:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-23T18:36:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.4", 2023-06-23T18:36:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDo5ZVkQAyLDxAAekbATinpByJ83JjzbSYkOQ==", 2023-06-23T18:36:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-23T18:36:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-23T18:36:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.5", 2023-06-23T18:36:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDr5ZVkFnUODxAArCgvE9RdF7FGM/U3JNYV+w==", 2023-06-23T18:36:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-23T18:36:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-23T18:36:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.6", 2023-06-23T18:36:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDu5ZVkW3PWDRAAQw0817RbYvN4IBcVFyhpnQ==", 2023-06-23T18:36:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-23T18:36:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-23T18:36:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.7", 2023-06-23T18:36:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDx5ZVkwfQjDBAADPpq9Rb1V8q5R1zXTBMbaw==", 2023-06-23T18:36:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-23T18:36:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-23T18:36:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.0", 2023-06-23T18:36:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDV5ZVkbyA+LRAA4TQMvihkdeAgh/w8764lVA==", 2023-06-23T18:36:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-23T18:36:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow r", 2023-06-23T18:36:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow rw", 2023-06-23T18:36:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow rwx" 2023-06-23T18:36:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.1", 2023-06-23T18:36:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDV5ZVkv4nPMxAAGMmXeX2s6ITjDkz5WmEAcA==", 2023-06-23T18:36:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.773 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-23T18:36:52.773 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow r", 2023-06-23T18:36:52.773 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow rw", 2023-06-23T18:36:52.773 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow rwx" 2023-06-23T18:36:52.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.775 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.2", 2023-06-23T18:36:52.775 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDV5ZVksOp9MBAAc9mALTNbDtZw8UO1fZNl9A==", 2023-06-23T18:36:52.775 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.775 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-23T18:36:52.776 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow r", 2023-06-23T18:36:52.776 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow rw", 2023-06-23T18:36:52.776 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow rwx" 2023-06-23T18:36:52.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.777 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.3", 2023-06-23T18:36:52.777 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDV5ZVkAYOHORAA8sPnGr3zVpU78Yc8Zz1z2w==", 2023-06-23T18:36:52.778 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.778 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-23T18:36:52.778 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow r", 2023-06-23T18:36:52.779 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow rw", 2023-06-23T18:36:52.779 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow rwx" 2023-06-23T18:36:52.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.780 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.admin", 2023-06-23T18:36:52.780 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDU5ZVkokFWMxAA9wbyXrZZzqxOOOx2pdCOZw==", 2023-06-23T18:36:52.781 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.781 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow *", 2023-06-23T18:36:52.781 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2023-06-23T18:36:52.782 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2023-06-23T18:36:52.782 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.782 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-mds", 2023-06-23T18:36:52.782 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQD35ZVkmmg+ChAAy4CuseYUQND5PuBTGkNz5g==", 2023-06-23T18:36:52.783 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.783 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-mds" 2023-06-23T18:36:52.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.783 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-mgr", 2023-06-23T18:36:52.783 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQD35ZVk03I+ChAAPPAYPYjXplgYAi4DtfZkCg==", 2023-06-23T18:36:52.783 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.784 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-mgr" 2023-06-23T18:36:52.784 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.784 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-osd", 2023-06-23T18:36:52.784 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQD35ZVkB3o+ChAA7LcEax9zsHtBCOT/1Xd22w==", 2023-06-23T18:36:52.784 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.785 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-osd" 2023-06-23T18:36:52.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.785 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-rbd", 2023-06-23T18:36:52.785 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQD35ZVk6YA+ChAANiP+ZWa4vV4gXH8npGBb/w==", 2023-06-23T18:36:52.785 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.786 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-rbd" 2023-06-23T18:36:52.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.786 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-06-23T18:36:52.786 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQD35ZVkCYg+ChAAvIdu/RNiNmDDEG5bZH7uaQ==", 2023-06-23T18:36:52.786 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.786 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-06-23T18:36:52.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.787 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-rgw", 2023-06-23T18:36:52.787 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQD35ZVkX48+ChAAHncUASeEXvbs5fGl6Rh0NQ==", 2023-06-23T18:36:52.787 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.787 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-rgw" 2023-06-23T18:36:52.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.788 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mgr.x", 2023-06-23T18:36:52.788 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDV5ZVkAsHsERAAFNSGaIjAfuHkhxdQi1vpjg==", 2023-06-23T18:36:52.788 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.788 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow *", 2023-06-23T18:36:52.788 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile mgr", 2023-06-23T18:36:52.789 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:52.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:52.789 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mgr.y", 2023-06-23T18:36:52.789 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDV5ZVkFO5vDRAAeqcx8oxKGghSCg1u4rWQ/A==", 2023-06-23T18:36:52.789 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-23T18:36:52.789 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow *", 2023-06-23T18:36:52.790 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile mgr", 2023-06-23T18:36:52.790 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-23T18:36:52.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:52.790 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-06-23T18:36:52.790 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-23T18:36:52.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.745+0000 7f331e926700 1 -- 172.21.15.79:0/2046642626 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f330406ace0 msgr2=0x7f330406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:52.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.745+0000 7f331e926700 1 --2- 172.21.15.79:0/2046642626 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f330406ace0 0x7f330406d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f330c000c00 tx=0x7f330c01c040 comp rx=0 tx=0).stop 2023-06-23T18:36:52.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.745+0000 7f331e926700 1 -- 172.21.15.79:0/2046642626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f33180aee30 msgr2=0x7f33180cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:52.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.745+0000 7f331e926700 1 --2- 172.21.15.79:0/2046642626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f33180aee30 0x7f33180cf850 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f330800c460 tx=0x7f330800c820 comp rx=0 tx=0).stop 2023-06-23T18:36:52.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.745+0000 7f331e926700 1 -- 172.21.15.79:0/2046642626 shutdown_connections 2023-06-23T18:36:52.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.745+0000 7f331e926700 1 --2- 172.21.15.79:0/2046642626 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f330406ace0 0x7f330406d190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.745+0000 7f331e926700 1 --2- 172.21.15.79:0/2046642626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33180b1270 0x7f3318152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.745+0000 7f331e926700 1 --2- 172.21.15.79:0/2046642626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3318071490 0x7f33180c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.745+0000 7f331e926700 1 --2- 172.21.15.79:0/2046642626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f33180aee30 0x7f33180cf850 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.745+0000 7f331e926700 1 -- 172.21.15.79:0/2046642626 >> 172.21.15.79:0/2046642626 conn(0x7f331806d660 msgr2=0x7f33180cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:52.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.745+0000 7f331e926700 1 -- 172.21.15.79:0/2046642626 shutdown_connections 2023-06-23T18:36:52.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.745+0000 7f331e926700 1 -- 172.21.15.79:0/2046642626 wait complete. 2023-06-23T18:36:52.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-06-23T18:36:52.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.910+0000 7f5dcf5ac700 1 Processor -- start 2023-06-23T18:36:52.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.910+0000 7f5dcf5ac700 1 -- start start 2023-06-23T18:36:52.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.911+0000 7f5dcf5ac700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dc80b83c0 0x7f5dc80cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:52.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.911+0000 7f5dcf5ac700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5dc80d3710 0x7f5dc80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:52.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.911+0000 7f5dcf5ac700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc80cfee0 con 0x7f5dc80cfa90 2023-06-23T18:36:52.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.911+0000 7f5dcf5ac700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc80d0a80 con 0x7f5dc80d3710 2023-06-23T18:36:52.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.911+0000 7f5dcf5ac700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc80b8790 con 0x7f5dc80b83c0 2023-06-23T18:36:52.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.911+0000 7f5dccb47700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5dc80d3710 0x7f5dc80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:52.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.911+0000 7f5dcdb49700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5dc80cfa90 0x7f5dc815feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44348/0 (socket says 172.21.15.79:44348) 2023-06-23T18:36:52.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.911+0000 7f5dcdb49700 1 -- 172.21.15.79:0/2284611769 learned_addr learned my addr 172.21.15.79:0/2284611769 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:52.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.911+0000 7f5dcd348700 1 --2- 172.21.15.79:0/2284611769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dc80b83c0 0x7f5dc80cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:52.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.911+0000 7f5dccb47700 1 -- 172.21.15.79:0/2284611769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dc80b83c0 msgr2=0x7f5dc80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:52.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.911+0000 7f5dccb47700 1 --2- 172.21.15.79:0/2284611769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dc80b83c0 0x7f5dc80cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.911+0000 7f5dccb47700 1 -- 172.21.15.79:0/2284611769 >> v1:172.21.15.79:6789/0 conn(0x7f5dc80cfa90 legacy=0x7f5dc815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:52.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.911+0000 7f5dccb47700 1 -- 172.21.15.79:0/2284611769 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc80b4e70 con 0x7f5dc80d3710 2023-06-23T18:36:52.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.911+0000 7f5dbffff700 1 -- 172.21.15.79:0/2284611769 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186363534 0 0) 0x7f5dc0001490 con 0x7f5dc80cfa90 2023-06-23T18:36:52.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.912+0000 7f5dccb47700 1 --2- 172.21.15.79:0/2284611769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc80d3710 0x7f5dc80cf550 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009840 tx=0x7f5db8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61091d01b6a5e11 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:52.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.912+0000 7f5dbffff700 1 -- 172.21.15.79:0/2284611769 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8011830 con 0x7f5dc80d3710 2023-06-23T18:36:52.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.912+0000 7f5dbffff700 1 -- 172.21.15.79:0/2284611769 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db8011990 con 0x7f5dc80d3710 2023-06-23T18:36:52.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.912+0000 7f5dbffff700 1 -- 172.21.15.79:0/2284611769 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8011d20 con 0x7f5dc80d3710 2023-06-23T18:36:52.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.912+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/2284611769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc80d3710 msgr2=0x7f5dc80cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:52.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.912+0000 7f5dcf5ac700 1 --2- 172.21.15.79:0/2284611769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc80d3710 0x7f5dc80cf550 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009840 tx=0x7f5db8009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:52.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.912+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/2284611769 shutdown_connections 2023-06-23T18:36:52.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.912+0000 7f5dcf5ac700 1 --2- 172.21.15.79:0/2284611769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dc80b83c0 0x7f5dc80cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.912+0000 7f5dcf5ac700 1 --2- 172.21.15.79:0/2284611769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc80d3710 0x7f5dc80cf550 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.912+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/2284611769 >> 172.21.15.79:0/2284611769 conn(0x7f5dc806b350 msgr2=0x7f5dc806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:52.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.912+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/2284611769 shutdown_connections 2023-06-23T18:36:52.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.912+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/2284611769 wait complete. 2023-06-23T18:36:52.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.914+0000 7f5dcf5ac700 1 Processor -- start 2023-06-23T18:36:52.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.914+0000 7f5dcf5ac700 1 -- start start 2023-06-23T18:36:52.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcf5ac700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc80b83c0 0x7f5dc8154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:52.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcf5ac700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dc80cfa90 0x7f5dc80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:52.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcf5ac700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dc80d3710 0x7f5dc8147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:52.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcf5ac700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc81551a0 con 0x7f5dc80b83c0 2023-06-23T18:36:52.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcf5ac700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc815ddd0 con 0x7f5dc80d3710 2023-06-23T18:36:52.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcf5ac700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc815e0b0 con 0x7f5dc80cfa90 2023-06-23T18:36:52.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcd348700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc80b83c0 0x7f5dc8154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:52.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcd348700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc80b83c0 0x7f5dc8154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41310/0 (socket says 172.21.15.79:41310) 2023-06-23T18:36:52.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcd348700 1 -- 172.21.15.79:0/3208665446 learned_addr learned my addr 172.21.15.79:0/3208665446 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:52.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcd348700 1 -- 172.21.15.79:0/3208665446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dc80cfa90 msgr2=0x7f5dc80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:52.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcd348700 1 --2- 172.21.15.79:0/3208665446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dc80cfa90 0x7f5dc80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcd348700 1 -- 172.21.15.79:0/3208665446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dc80d3710 msgr2=0x7f5dc8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:52.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcd348700 1 --2- 172.21.15.79:0/3208665446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dc80d3710 0x7f5dc8147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:52.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcd348700 1 -- 172.21.15.79:0/3208665446 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db80094a0 con 0x7f5dc80b83c0 2023-06-23T18:36:52.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.915+0000 7f5dcd348700 1 --2- 172.21.15.79:0/3208665446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc80b83c0 0x7f5dc8154c60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400d800 tx=0x7f5dc400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:52.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.916+0000 7f5dbdffb700 1 -- 172.21.15.79:0/3208665446 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40125f0 con 0x7f5dc80b83c0 2023-06-23T18:36:52.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.916+0000 7f5dbdffb700 1 -- 172.21.15.79:0/3208665446 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc4012750 con 0x7f5dc80b83c0 2023-06-23T18:36:52.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.916+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/3208665446 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc815e390 con 0x7f5dc80b83c0 2023-06-23T18:36:52.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.916+0000 7f5dbdffb700 1 -- 172.21.15.79:0/3208665446 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4012b00 con 0x7f5dc80b83c0 2023-06-23T18:36:52.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.916+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/3208665446 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc815e950 con 0x7f5dc80b83c0 2023-06-23T18:36:52.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.917+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/3208665446 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc815eae0 con 0x7f5dc80b83c0 2023-06-23T18:36:52.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.917+0000 7f5dbdffb700 1 -- 172.21.15.79:0/3208665446 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5dc4012c60 con 0x7f5dc80b83c0 2023-06-23T18:36:52.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.917+0000 7f5dbdffb700 1 --2- 172.21.15.79:0/3208665446 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5dac06ace0 0x7f5dac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:52.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.917+0000 7f5dbdffb700 1 -- 172.21.15.79:0/3208665446 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5dc4089490 con 0x7f5dc80b83c0 2023-06-23T18:36:52.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.917+0000 7f5dccb47700 1 --2- 172.21.15.79:0/3208665446 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5dac06ace0 0x7f5dac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:52.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.918+0000 7f5dccb47700 1 --2- 172.21.15.79:0/3208665446 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5dac06ace0 0x7f5dac06d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80d3b20 tx=0x7f5db8010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:52.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:52.922+0000 7f5dbdffb700 1 -- 172.21.15.79:0/3208665446 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5dc4054920 con 0x7f5dc80b83c0 2023-06-23T18:36:53.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.188+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/3208665446 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f5dc80b7410 con 0x7f5dc80b83c0 2023-06-23T18:36:53.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.194+0000 7f5dbdffb700 1 -- 172.21.15.79:0/3208665446 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5dc4057f40 con 0x7f5dc80b83c0 2023-06-23T18:36:53.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.196+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/3208665446 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5dac06ace0 msgr2=0x7f5dac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:53.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.196+0000 7f5dcf5ac700 1 --2- 172.21.15.79:0/3208665446 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5dac06ace0 0x7f5dac06d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80d3b20 tx=0x7f5db8010000 comp rx=0 tx=0).stop 2023-06-23T18:36:53.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.196+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/3208665446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc80b83c0 msgr2=0x7f5dc8154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:53.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.196+0000 7f5dcf5ac700 1 --2- 172.21.15.79:0/3208665446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc80b83c0 0x7f5dc8154c60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400d800 tx=0x7f5dc400dbc0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.196+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/3208665446 shutdown_connections 2023-06-23T18:36:53.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.196+0000 7f5dcf5ac700 1 --2- 172.21.15.79:0/3208665446 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5dac06ace0 0x7f5dac06d190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.196+0000 7f5dcf5ac700 1 --2- 172.21.15.79:0/3208665446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dc80d3710 0x7f5dc8147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.196+0000 7f5dcf5ac700 1 --2- 172.21.15.79:0/3208665446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc80b83c0 0x7f5dc8154c60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.196+0000 7f5dcf5ac700 1 --2- 172.21.15.79:0/3208665446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dc80cfa90 0x7f5dc80b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.196+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/3208665446 >> 172.21.15.79:0/3208665446 conn(0x7f5dc806b350 msgr2=0x7f5dc80b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:53.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.196+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/3208665446 shutdown_connections 2023-06-23T18:36:53.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.196+0000 7f5dcf5ac700 1 -- 172.21.15.79:0/3208665446 wait complete. 2023-06-23T18:36:53.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-06-23T18:36:53.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.328+0000 7fd290149700 1 Processor -- start 2023-06-23T18:36:53.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.329+0000 7fd290149700 1 -- start start 2023-06-23T18:36:53.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.329+0000 7fd290149700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd288071490 0x7fd288071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:53.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.329+0000 7fd290149700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2881608f0 con 0x7fd2880aee30 2023-06-23T18:36:53.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.329+0000 7fd290149700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd288160e20 con 0x7fd2880b1270 2023-06-23T18:36:53.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.329+0000 7fd290149700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28806acb0 con 0x7fd288071490 2023-06-23T18:36:53.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28e6e6700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd2880aee30 0x7fd28806a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44364/0 (socket says 172.21.15.79:44364) 2023-06-23T18:36:53.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28e6e6700 1 -- 172.21.15.79:0/3058672266 learned_addr learned my addr 172.21.15.79:0/3058672266 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:53.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28dee5700 1 --2- 172.21.15.79:0/3058672266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd288071490 0x7fd288071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:53.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838574972 0 0) 0x7fd2881608f0 con 0x7fd2880aee30 2023-06-23T18:36:53.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2740035b0 con 0x7fd2880aee30 2023-06-23T18:36:53.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209894737 0 0) 0x7fd288160e20 con 0x7fd2880b1270 2023-06-23T18:36:53.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2881608f0 con 0x7fd2880b1270 2023-06-23T18:36:53.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3451678318 0 0) 0x7fd2740035b0 con 0x7fd2880aee30 2023-06-23T18:36:53.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd288071490 msgr2=0x7fd288071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:53.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 --2- 172.21.15.79:0/3058672266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd288071490 0x7fd288071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 >> v1:172.21.15.155:6789/0 conn(0x7fd2880b1270 legacy=0x7fd288069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:53.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2880cd4f0 con 0x7fd2880aee30 2023-06-23T18:36:53.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fd2800019f0 con 0x7fd2880aee30 2023-06-23T18:36:53.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 >> v1:172.21.15.79:6789/0 conn(0x7fd2880aee30 legacy=0x7fd28806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:53.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 --2- 172.21.15.79:0/3058672266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd274003bc0 0x7fd274003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:53.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 --2- 172.21.15.79:0/3058672266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd274004850 0x7fd274004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:53.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 --2- 172.21.15.79:0/3058672266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd274005200 0x7fd27400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:53.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28806acb0 con 0x7fd274003bc0 2023-06-23T18:36:53.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27400c060 con 0x7fd274004850 2023-06-23T18:36:53.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27400c220 con 0x7fd274005200 2023-06-23T18:36:53.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28dee5700 1 --2- 172.21.15.79:0/3058672266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd288071490 0x7fd288071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:53.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28dee5700 1 --2- 172.21.15.79:0/3058672266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd274003bc0 0x7fd274003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:53.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28e6e6700 1 --2- 172.21.15.79:0/3058672266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd274005200 0x7fd27400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:53.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.330+0000 7fd28d6e4700 1 --2- 172.21.15.79:0/3058672266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd274004850 0x7fd274004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:53.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.331+0000 7fd28dee5700 1 -- 172.21.15.79:0/3058672266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd274005200 msgr2=0x7fd27400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:53.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.331+0000 7fd28dee5700 1 --2- 172.21.15.79:0/3058672266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd274005200 0x7fd27400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.331+0000 7fd28dee5700 1 -- 172.21.15.79:0/3058672266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd274004850 msgr2=0x7fd274004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:53.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.331+0000 7fd28dee5700 1 --2- 172.21.15.79:0/3058672266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd274004850 0x7fd274004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.331+0000 7fd28dee5700 1 -- 172.21.15.79:0/3058672266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2740035b0 con 0x7fd274003bc0 2023-06-23T18:36:53.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.331+0000 7fd28dee5700 1 --2- 172.21.15.79:0/3058672266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd274003bc0 0x7fd274003fb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd2880cd2e0 tx=0x7fd274005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80b92961f5deb55 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:53.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.331+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278004030 con 0x7fd274003bc0 2023-06-23T18:36:53.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.331+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd278004190 con 0x7fd274003bc0 2023-06-23T18:36:53.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.331+0000 7fd28cee3700 1 -- 172.21.15.79:0/3058672266 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278004540 con 0x7fd274003bc0 2023-06-23T18:36:53.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.331+0000 7fd290149700 1 -- 172.21.15.79:0/3058672266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd274003bc0 msgr2=0x7fd274003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:53.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.331+0000 7fd290149700 1 --2- 172.21.15.79:0/3058672266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd274003bc0 0x7fd274003fb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd2880cd2e0 tx=0x7fd274005880 comp rx=0 tx=0).stop 2023-06-23T18:36:53.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.331+0000 7fd28e6e6700 1 -- 172.21.15.79:0/3058672266 reap_dead start 2023-06-23T18:36:53.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.331+0000 7fd290149700 1 -- 172.21.15.79:0/3058672266 shutdown_connections 2023-06-23T18:36:53.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.331+0000 7fd290149700 1 -- 172.21.15.79:0/3058672266 >> 172.21.15.79:0/3058672266 conn(0x7fd28806d660 msgr2=0x7fd28806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:53.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.332+0000 7fd290149700 1 -- 172.21.15.79:0/3058672266 shutdown_connections 2023-06-23T18:36:53.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.332+0000 7fd290149700 1 -- 172.21.15.79:0/3058672266 wait complete. 2023-06-23T18:36:53.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.333+0000 7fd290149700 1 Processor -- start 2023-06-23T18:36:53.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.333+0000 7fd290149700 1 -- start start 2023-06-23T18:36:53.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd290149700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd274003bc0 0x7fd2880c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:53.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd290149700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2880acd10 0x7fd2880ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:53.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd290149700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2880ae590 0x7fd288155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:53.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd290149700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880abc90 con 0x7fd2880acd10 2023-06-23T18:36:53.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd290149700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880abf70 con 0x7fd2880ae590 2023-06-23T18:36:53.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd290149700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880ac250 con 0x7fd274003bc0 2023-06-23T18:36:53.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd28d6e4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2880acd10 0x7fd2880ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:53.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd28d6e4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2880acd10 0x7fd2880ae050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41330/0 (socket says 172.21.15.79:41330) 2023-06-23T18:36:53.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd28d6e4700 1 -- 172.21.15.79:0/1135442747 learned_addr learned my addr 172.21.15.79:0/1135442747 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:53.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd28e6e6700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2880ae590 0x7fd288155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:53.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd28dee5700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd274003bc0 0x7fd2880c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:53.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd28d6e4700 1 -- 172.21.15.79:0/1135442747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd274003bc0 msgr2=0x7fd2880c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:53.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd28d6e4700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd274003bc0 0x7fd2880c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd28d6e4700 1 -- 172.21.15.79:0/1135442747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2880ae590 msgr2=0x7fd288155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:53.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd28d6e4700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2880ae590 0x7fd288155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd28d6e4700 1 -- 172.21.15.79:0/1135442747 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2880ac530 con 0x7fd2880acd10 2023-06-23T18:36:53.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.334+0000 7fd28e6e6700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2880ae590 0x7fd288155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:53.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.335+0000 7fd28d6e4700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2880acd10 0x7fd2880ae050 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd28400f770 tx=0x7fd28400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:53.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.335+0000 7fd27e7fc700 1 -- 172.21.15.79:0/1135442747 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2840052e0 con 0x7fd2880acd10 2023-06-23T18:36:53.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.335+0000 7fd27e7fc700 1 -- 172.21.15.79:0/1135442747 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd284005460 con 0x7fd2880acd10 2023-06-23T18:36:53.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.335+0000 7fd290149700 1 -- 172.21.15.79:0/1135442747 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2880ac810 con 0x7fd2880acd10 2023-06-23T18:36:53.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.335+0000 7fd27e7fc700 1 -- 172.21.15.79:0/1135442747 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2840038c0 con 0x7fd2880acd10 2023-06-23T18:36:53.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.335+0000 7fd290149700 1 -- 172.21.15.79:0/1135442747 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2880c5cd0 con 0x7fd2880acd10 2023-06-23T18:36:53.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.336+0000 7fd290149700 1 -- 172.21.15.79:0/1135442747 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2880ccc50 con 0x7fd2880acd10 2023-06-23T18:36:53.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.336+0000 7fd27e7fc700 1 -- 172.21.15.79:0/1135442747 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd284003a20 con 0x7fd2880acd10 2023-06-23T18:36:53.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.336+0000 7fd27e7fc700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd26c06ace0 0x7fd26c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:53.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.336+0000 7fd27e7fc700 1 -- 172.21.15.79:0/1135442747 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd284089060 con 0x7fd2880acd10 2023-06-23T18:36:53.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.336+0000 7fd28dee5700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd26c06ace0 0x7fd26c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:53.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.337+0000 7fd28dee5700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd26c06ace0 0x7fd26c06d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd2880cd2e0 tx=0x7fd278008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:53.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.341+0000 7fd27e7fc700 1 -- 172.21.15.79:0/1135442747 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd284054460 con 0x7fd2880acd10 2023-06-23T18:36:53.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.605+0000 7fd290149700 1 -- 172.21.15.79:0/1135442747 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fd2880724a0 con 0x7fd2880acd10 2023-06-23T18:36:53.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.609+0000 7fd27e7fc700 1 -- 172.21.15.79:0/1135442747 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd284057a80 con 0x7fd2880acd10 2023-06-23T18:36:53.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.611+0000 7fd290149700 1 -- 172.21.15.79:0/1135442747 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd26c06ace0 msgr2=0x7fd26c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:53.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.611+0000 7fd290149700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd26c06ace0 0x7fd26c06d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd2880cd2e0 tx=0x7fd278008070 comp rx=0 tx=0).stop 2023-06-23T18:36:53.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.611+0000 7fd290149700 1 -- 172.21.15.79:0/1135442747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2880acd10 msgr2=0x7fd2880ae050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:53.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.611+0000 7fd290149700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2880acd10 0x7fd2880ae050 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd28400f770 tx=0x7fd28400fb30 comp rx=0 tx=0).stop 2023-06-23T18:36:53.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.611+0000 7fd290149700 1 -- 172.21.15.79:0/1135442747 shutdown_connections 2023-06-23T18:36:53.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.611+0000 7fd290149700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd26c06ace0 0x7fd26c06d190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.611+0000 7fd290149700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2880ae590 0x7fd288155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.611+0000 7fd290149700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd274003bc0 0x7fd2880c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.611+0000 7fd290149700 1 --2- 172.21.15.79:0/1135442747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2880acd10 0x7fd2880ae050 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.611+0000 7fd290149700 1 -- 172.21.15.79:0/1135442747 >> 172.21.15.79:0/1135442747 conn(0x7fd28806d660 msgr2=0x7fd2880cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:53.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.611+0000 7fd290149700 1 -- 172.21.15.79:0/1135442747 shutdown_connections 2023-06-23T18:36:53.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.612+0000 7fd290149700 1 -- 172.21.15.79:0/1135442747 wait complete. 2023-06-23T18:36:53.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-06-23T18:36:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.744+0000 7f8cdbb9f700 1 Processor -- start 2023-06-23T18:36:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.744+0000 7f8cdbb9f700 1 -- start start 2023-06-23T18:36:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cdbb9f700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8cd40b1270 0x7f8cd415c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cdbb9f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd40aee30 0x7f8cd415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cdbb9f700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd41608f0 con 0x7f8cd4071490 2023-06-23T18:36:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cdbb9f700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd4160e20 con 0x7f8cd40b1270 2023-06-23T18:36:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cdbb9f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40cd4f0 con 0x7f8cd40aee30 2023-06-23T18:36:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cda13c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd40aee30 0x7f8cd415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:53.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cd913a700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8cd40b1270 0x7f8cd415c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:53.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cd913a700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8cd40b1270 0x7f8cd415c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:47566/0 (socket says 172.21.15.79:47566) 2023-06-23T18:36:53.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cd993b700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f8cd4071490 0x7f8cd4071860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:52648/0 (socket says 172.21.15.79:52648) 2023-06-23T18:36:53.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cda13c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd40aee30 0x7f8cd415cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:34112/0 (socket says 172.21.15.79:34112) 2023-06-23T18:36:53.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cd913a700 1 -- 172.21.15.79:0/3087486415 learned_addr learned my addr 172.21.15.79:0/3087486415 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:53.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cd913a700 1 -- 172.21.15.79:0/3087486415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd40aee30 msgr2=0x7f8cd415cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:53.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cd913a700 1 --2- 172.21.15.79:0/3087486415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd40aee30 0x7f8cd415cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cd913a700 1 -- 172.21.15.79:0/3087486415 >> v1:172.21.15.155:6789/0 conn(0x7f8cd4071490 legacy=0x7f8cd4071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:53.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cd913a700 1 -- 172.21.15.79:0/3087486415 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd4160b40 con 0x7f8cd40b1270 2023-06-23T18:36:53.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.745+0000 7f8cda13c700 1 --2- 172.21.15.79:0/3087486415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd40aee30 0x7f8cd415cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:53.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.746+0000 7f8cd913a700 1 --2- 172.21.15.79:0/3087486415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd40b1270 0x7f8cd415c9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40098f0 tx=0x7f8cc4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=154bf79c07aabe97 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:53.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.746+0000 7f8cd8939700 1 -- 172.21.15.79:0/3087486415 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4011830 con 0x7f8cd40b1270 2023-06-23T18:36:53.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.746+0000 7f8cd8939700 1 -- 172.21.15.79:0/3087486415 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc4011990 con 0x7f8cd40b1270 2023-06-23T18:36:53.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.746+0000 7f8cd8939700 1 -- 172.21.15.79:0/3087486415 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4011d20 con 0x7f8cd40b1270 2023-06-23T18:36:53.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.746+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/3087486415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd40b1270 msgr2=0x7f8cd415c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:53.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.746+0000 7f8cdbb9f700 1 --2- 172.21.15.79:0/3087486415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd40b1270 0x7f8cd415c9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40098f0 tx=0x7f8cc4009c00 comp rx=0 tx=0).stop 2023-06-23T18:36:53.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.746+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/3087486415 shutdown_connections 2023-06-23T18:36:53.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.746+0000 7f8cdbb9f700 1 --2- 172.21.15.79:0/3087486415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd40aee30 0x7f8cd415cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.746+0000 7f8cdbb9f700 1 --2- 172.21.15.79:0/3087486415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd40b1270 0x7f8cd415c9f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.746+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/3087486415 >> 172.21.15.79:0/3087486415 conn(0x7f8cd406d660 msgr2=0x7f8cd406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:53.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.746+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/3087486415 shutdown_connections 2023-06-23T18:36:53.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.746+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/3087486415 wait complete. 2023-06-23T18:36:53.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.748+0000 7f8cdbb9f700 1 Processor -- start 2023-06-23T18:36:53.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.748+0000 7f8cdbb9f700 1 -- start start 2023-06-23T18:36:53.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.748+0000 7f8cdbb9f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd4071490 0x7f8cd40b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:53.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.748+0000 7f8cdbb9f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cd40aee30 0x7f8cd40b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:53.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.748+0000 7f8cdbb9f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd40b1270 0x7f8cd40c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:53.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.748+0000 7f8cdbb9f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40c6160 con 0x7f8cd40aee30 2023-06-23T18:36:53.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.748+0000 7f8cdbb9f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd4154ab0 con 0x7f8cd4071490 2023-06-23T18:36:53.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.748+0000 7f8cdbb9f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40b2b60 con 0x7f8cd40b1270 2023-06-23T18:36:53.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.748+0000 7f8cd913a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cd40aee30 0x7f8cd40b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:53.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.748+0000 7f8cd913a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cd40aee30 0x7f8cd40b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41346/0 (socket says 172.21.15.79:41346) 2023-06-23T18:36:53.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.748+0000 7f8cd913a700 1 -- 172.21.15.79:0/2291804277 learned_addr learned my addr 172.21.15.79:0/2291804277 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:53.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.749+0000 7f8cd993b700 1 --2- 172.21.15.79:0/2291804277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd4071490 0x7f8cd40b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:53.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.749+0000 7f8cda13c700 1 --2- 172.21.15.79:0/2291804277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd40b1270 0x7f8cd40c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.749+0000 7f8cd913a700 1 -- 172.21.15.79:0/2291804277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd40b1270 msgr2=0x7f8cd40c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.749+0000 7f8cd913a700 1 --2- 172.21.15.79:0/2291804277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd40b1270 0x7f8cd40c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.749+0000 7f8cd913a700 1 -- 172.21.15.79:0/2291804277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd4071490 msgr2=0x7f8cd40b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.749+0000 7f8cd913a700 1 --2- 172.21.15.79:0/2291804277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd4071490 0x7f8cd40b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.749+0000 7f8cd913a700 1 -- 172.21.15.79:0/2291804277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc009a70 con 0x7f8cd40aee30 2023-06-23T18:36:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.749+0000 7f8cd913a700 1 --2- 172.21.15.79:0/2291804277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cd40aee30 0x7f8cd40b2620 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4014040 tx=0x7f8cc4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.749+0000 7f8cca7fc700 1 -- 172.21.15.79:0/2291804277 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40044d0 con 0x7f8cd40aee30 2023-06-23T18:36:53.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.749+0000 7f8cca7fc700 1 -- 172.21.15.79:0/2291804277 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc401e070 con 0x7f8cd40aee30 2023-06-23T18:36:53.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.749+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/2291804277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc4009550 con 0x7f8cd40aee30 2023-06-23T18:36:53.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.749+0000 7f8cca7fc700 1 -- 172.21.15.79:0/2291804277 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40038e0 con 0x7f8cd40aee30 2023-06-23T18:36:53.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.749+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/2291804277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd40b5410 con 0x7f8cd40aee30 2023-06-23T18:36:53.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.750+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/2291804277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd40724a0 con 0x7f8cd40aee30 2023-06-23T18:36:53.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.750+0000 7f8cca7fc700 1 -- 172.21.15.79:0/2291804277 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8cc4003a40 con 0x7f8cd40aee30 2023-06-23T18:36:53.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.751+0000 7f8cca7fc700 1 --2- 172.21.15.79:0/2291804277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cb806ace0 0x7f8cb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:53.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.751+0000 7f8cca7fc700 1 -- 172.21.15.79:0/2291804277 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8cc4089e80 con 0x7f8cd40aee30 2023-06-23T18:36:53.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.751+0000 7f8cd993b700 1 --2- 172.21.15.79:0/2291804277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cb806ace0 0x7f8cb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:53.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.751+0000 7f8cd993b700 1 --2- 172.21.15.79:0/2291804277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cb806ace0 0x7f8cb806d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0000f60 tx=0x7f8cd0007ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:53.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:53.755+0000 7f8cca7fc700 1 -- 172.21.15.79:0/2291804277 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8cc4055310 con 0x7f8cd40aee30 2023-06-23T18:36:54.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.017+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/2291804277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f8cd40aeb60 con 0x7f8cd40aee30 2023-06-23T18:36:54.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.021+0000 7f8cca7fc700 1 -- 172.21.15.79:0/2291804277 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8cc40200a0 con 0x7f8cd40aee30 2023-06-23T18:36:54.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.023+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/2291804277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cb806ace0 msgr2=0x7f8cb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.023+0000 7f8cdbb9f700 1 --2- 172.21.15.79:0/2291804277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cb806ace0 0x7f8cb806d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0000f60 tx=0x7f8cd0007ed0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.023+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/2291804277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cd40aee30 msgr2=0x7f8cd40b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.023+0000 7f8cdbb9f700 1 --2- 172.21.15.79:0/2291804277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cd40aee30 0x7f8cd40b2620 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4014040 tx=0x7f8cc4004750 comp rx=0 tx=0).stop 2023-06-23T18:36:54.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.023+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/2291804277 shutdown_connections 2023-06-23T18:36:54.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.023+0000 7f8cdbb9f700 1 --2- 172.21.15.79:0/2291804277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd40b1270 0x7f8cd40c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.023+0000 7f8cdbb9f700 1 --2- 172.21.15.79:0/2291804277 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cb806ace0 0x7f8cb806d190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.023+0000 7f8cdbb9f700 1 --2- 172.21.15.79:0/2291804277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd4071490 0x7f8cd40b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.023+0000 7f8cdbb9f700 1 --2- 172.21.15.79:0/2291804277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cd40aee30 0x7f8cd40b2620 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.023+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/2291804277 >> 172.21.15.79:0/2291804277 conn(0x7f8cd406d660 msgr2=0x7f8cd40ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:54.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.023+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/2291804277 shutdown_connections 2023-06-23T18:36:54.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.023+0000 7f8cdbb9f700 1 -- 172.21.15.79:0/2291804277 wait complete. 2023-06-23T18:36:54.040 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-06-23T18:36:54.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.157+0000 7f68a8e84700 1 Processor -- start 2023-06-23T18:36:54.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.157+0000 7f68a8e84700 1 -- start start 2023-06-23T18:36:54.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a8e84700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a4071490 0x7f68a4071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a8e84700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68a40b1270 0x7f68a4071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a8e84700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f68a40aee30 0x7f68a4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a8e84700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a41608f0 con 0x7f68a40b1270 2023-06-23T18:36:54.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a8e84700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a40722e0 con 0x7f68a40aee30 2023-06-23T18:36:54.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a8e84700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a4160e20 con 0x7f68a4071490 2023-06-23T18:36:54.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a1d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68a40b1270 0x7f68a4071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a1d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68a40b1270 0x7f68a4071da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41352/0 (socket says 172.21.15.79:41352) 2023-06-23T18:36:54.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a1d9b700 1 -- 172.21.15.79:0/2900896078 learned_addr learned my addr 172.21.15.79:0/2900896078 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:54.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a2d9d700 1 --2- 172.21.15.79:0/2900896078 >> v2:172.21.15.155:3300/0 conn(0x7f68a40aee30 0x7f68a4069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a259c700 1 --2- 172.21.15.79:0/2900896078 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a4071490 0x7f68a4071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a1d9b700 1 -- 172.21.15.79:0/2900896078 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a4071490 msgr2=0x7f68a4071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:54.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a1d9b700 1 --2- 172.21.15.79:0/2900896078 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a4071490 0x7f68a4071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a1d9b700 1 -- 172.21.15.79:0/2900896078 >> v2:172.21.15.155:3300/0 conn(0x7f68a40aee30 msgr2=0x7f68a4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:54.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a1d9b700 1 --2- 172.21.15.79:0/2900896078 >> v2:172.21.15.155:3300/0 conn(0x7f68a40aee30 0x7f68a4069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a1d9b700 1 -- 172.21.15.79:0/2900896078 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a40cd4f0 con 0x7f68a40b1270 2023-06-23T18:36:54.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a2d9d700 1 --2- 172.21.15.79:0/2900896078 >> v2:172.21.15.155:3300/0 conn(0x7f68a40aee30 0x7f68a4069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:54.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a259c700 1 --2- 172.21.15.79:0/2900896078 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a4071490 0x7f68a4071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:54.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a1d9b700 1 --2- 172.21.15.79:0/2900896078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68a40b1270 0x7f68a4071da0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f68a40cea10 tx=0x7f6898009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937fa943f6871c7f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:54.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.158+0000 7f68a159a700 1 -- 172.21.15.79:0/2900896078 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689800b870 con 0x7f68a40b1270 2023-06-23T18:36:54.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.159+0000 7f68a159a700 1 -- 172.21.15.79:0/2900896078 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689800b9d0 con 0x7f68a40b1270 2023-06-23T18:36:54.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.159+0000 7f68a159a700 1 -- 172.21.15.79:0/2900896078 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689800bd60 con 0x7f68a40b1270 2023-06-23T18:36:54.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.159+0000 7f68a8e84700 1 -- 172.21.15.79:0/2900896078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68a40b1270 msgr2=0x7f68a4071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.159+0000 7f68a8e84700 1 --2- 172.21.15.79:0/2900896078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68a40b1270 0x7f68a4071da0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f68a40cea10 tx=0x7f6898009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.159+0000 7f68a8e84700 1 -- 172.21.15.79:0/2900896078 shutdown_connections 2023-06-23T18:36:54.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.159+0000 7f68a8e84700 1 --2- 172.21.15.79:0/2900896078 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a4071490 0x7f68a4071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.159+0000 7f68a8e84700 1 --2- 172.21.15.79:0/2900896078 >> v2:172.21.15.155:3300/0 conn(0x7f68a40aee30 0x7f68a4069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.159+0000 7f68a8e84700 1 --2- 172.21.15.79:0/2900896078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68a40b1270 0x7f68a4071da0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.159+0000 7f68a8e84700 1 -- 172.21.15.79:0/2900896078 >> 172.21.15.79:0/2900896078 conn(0x7f68a406d660 msgr2=0x7f68a406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:54.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.159+0000 7f68a8e84700 1 -- 172.21.15.79:0/2900896078 shutdown_connections 2023-06-23T18:36:54.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.159+0000 7f68a8e84700 1 -- 172.21.15.79:0/2900896078 wait complete. 2023-06-23T18:36:54.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.160+0000 7f68a8e84700 1 Processor -- start 2023-06-23T18:36:54.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.161+0000 7f68a8e84700 1 -- start start 2023-06-23T18:36:54.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.161+0000 7f68a8e84700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a4071490 0x7f68a4146a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.161+0000 7f68a8e84700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68a40aee30 0x7f68a40c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.161+0000 7f68a8e84700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a40b1270 0x7f68a40cf510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.161+0000 7f68a8e84700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a40b48d0 con 0x7f68a40aee30 2023-06-23T18:36:54.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.161+0000 7f68a8e84700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a40b5450 con 0x7f68a4071490 2023-06-23T18:36:54.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.161+0000 7f68a8e84700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a4146fa0 con 0x7f68a40b1270 2023-06-23T18:36:54.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.161+0000 7f68a1d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68a40aee30 0x7f68a40c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.161+0000 7f68a1d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68a40aee30 0x7f68a40c90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41360/0 (socket says 172.21.15.79:41360) 2023-06-23T18:36:54.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.161+0000 7f68a1d9b700 1 -- 172.21.15.79:0/2227188028 learned_addr learned my addr 172.21.15.79:0/2227188028 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:54.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.161+0000 7f68a259c700 1 --2- 172.21.15.79:0/2227188028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a4071490 0x7f68a4146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.162+0000 7f68a2d9d700 1 --2- 172.21.15.79:0/2227188028 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a40b1270 0x7f68a40cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.162+0000 7f68a1d9b700 1 -- 172.21.15.79:0/2227188028 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a40b1270 msgr2=0x7f68a40cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.162+0000 7f68a1d9b700 1 --2- 172.21.15.79:0/2227188028 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a40b1270 0x7f68a40cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.162+0000 7f68a1d9b700 1 -- 172.21.15.79:0/2227188028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a4071490 msgr2=0x7f68a4146a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.162+0000 7f68a1d9b700 1 --2- 172.21.15.79:0/2227188028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a4071490 0x7f68a4146a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.162+0000 7f68a1d9b700 1 -- 172.21.15.79:0/2227188028 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6898009550 con 0x7f68a40aee30 2023-06-23T18:36:54.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.162+0000 7f68a1d9b700 1 --2- 172.21.15.79:0/2227188028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68a40aee30 0x7f68a40c90e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f68a4070bc0 tx=0x7f6898004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:54.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.162+0000 7f6892ffd700 1 -- 172.21.15.79:0/2227188028 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898004310 con 0x7f68a40aee30 2023-06-23T18:36:54.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.162+0000 7f6892ffd700 1 -- 172.21.15.79:0/2227188028 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689801e070 con 0x7f68a40aee30 2023-06-23T18:36:54.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.162+0000 7f6892ffd700 1 -- 172.21.15.79:0/2227188028 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68980036a0 con 0x7f68a40aee30 2023-06-23T18:36:54.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.162+0000 7f68a8e84700 1 -- 172.21.15.79:0/2227188028 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a40b4b20 con 0x7f68a40aee30 2023-06-23T18:36:54.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.162+0000 7f68a8e84700 1 -- 172.21.15.79:0/2227188028 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a40b50e0 con 0x7f68a40aee30 2023-06-23T18:36:54.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.163+0000 7f68a8e84700 1 -- 172.21.15.79:0/2227188028 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a40b5590 con 0x7f68a40aee30 2023-06-23T18:36:54.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.164+0000 7f6892ffd700 1 -- 172.21.15.79:0/2227188028 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6898020030 con 0x7f68a40aee30 2023-06-23T18:36:54.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.164+0000 7f6892ffd700 1 --2- 172.21.15.79:0/2227188028 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f688406ad10 0x7f688406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.164+0000 7f6892ffd700 1 -- 172.21.15.79:0/2227188028 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f689808a150 con 0x7f68a40aee30 2023-06-23T18:36:54.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.164+0000 7f68a259c700 1 --2- 172.21.15.79:0/2227188028 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f688406ad10 0x7f688406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.164+0000 7f68a259c700 1 --2- 172.21.15.79:0/2227188028 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f688406ad10 0x7f688406d1c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6894006fd0 tx=0x7f6894008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:54.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.168+0000 7f6892ffd700 1 -- 172.21.15.79:0/2227188028 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f68a40b5590 con 0x7f68a40aee30 2023-06-23T18:36:54.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.429+0000 7f68a8e84700 1 -- 172.21.15.79:0/2227188028 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f68a40b5590 con 0x7f68a40aee30 2023-06-23T18:36:54.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.433+0000 7f6892ffd700 1 -- 172.21.15.79:0/2227188028 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f68a40b5590 con 0x7f68a40aee30 2023-06-23T18:36:54.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.435+0000 7f68a8e84700 1 -- 172.21.15.79:0/2227188028 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f688406ad10 msgr2=0x7f688406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.435+0000 7f68a8e84700 1 --2- 172.21.15.79:0/2227188028 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f688406ad10 0x7f688406d1c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6894006fd0 tx=0x7f6894008040 comp rx=0 tx=0).stop 2023-06-23T18:36:54.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.435+0000 7f68a8e84700 1 -- 172.21.15.79:0/2227188028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68a40aee30 msgr2=0x7f68a40c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.435+0000 7f68a8e84700 1 --2- 172.21.15.79:0/2227188028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68a40aee30 0x7f68a40c90e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f68a4070bc0 tx=0x7f6898004750 comp rx=0 tx=0).stop 2023-06-23T18:36:54.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.435+0000 7f68a8e84700 1 -- 172.21.15.79:0/2227188028 shutdown_connections 2023-06-23T18:36:54.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.435+0000 7f68a8e84700 1 --2- 172.21.15.79:0/2227188028 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a40b1270 0x7f68a40cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.435+0000 7f68a8e84700 1 --2- 172.21.15.79:0/2227188028 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f688406ad10 0x7f688406d1c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.435+0000 7f68a8e84700 1 --2- 172.21.15.79:0/2227188028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a4071490 0x7f68a4146a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.435+0000 7f68a8e84700 1 --2- 172.21.15.79:0/2227188028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68a40aee30 0x7f68a40c90e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.435+0000 7f68a8e84700 1 -- 172.21.15.79:0/2227188028 >> 172.21.15.79:0/2227188028 conn(0x7f68a406d660 msgr2=0x7f68a41575c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:54.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.435+0000 7f68a8e84700 1 -- 172.21.15.79:0/2227188028 shutdown_connections 2023-06-23T18:36:54.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.435+0000 7f68a8e84700 1 -- 172.21.15.79:0/2227188028 wait complete. 2023-06-23T18:36:54.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-06-23T18:36:54.572 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.567+0000 7fa4f14a7700 1 Processor -- start 2023-06-23T18:36:54.572 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.567+0000 7fa4f14a7700 1 -- start start 2023-06-23T18:36:54.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4f14a7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0d3710 0x7fa4ec0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4f14a7700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa4ec0cfa90 0x7fa4ec0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4f14a7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ec0cd910 con 0x7fa4ec0b83c0 2023-06-23T18:36:54.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4f14a7700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4ec0cfee0 con 0x7fa4ec0cfa90 2023-06-23T18:36:54.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4f14a7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec0d0a80 con 0x7fa4ec0d3710 2023-06-23T18:36:54.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4eaffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa4ec0b83c0 0x7fa4ec0cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44408/0 (socket says 172.21.15.79:44408) 2023-06-23T18:36:54.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4eaffd700 1 -- 172.21.15.79:0/4187824149 learned_addr learned my addr 172.21.15.79:0/4187824149 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:54.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4eb7fe700 1 --2- 172.21.15.79:0/4187824149 >> v2:172.21.15.155:3300/0 conn(0x7fa4ec0cfa90 0x7fa4ec0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4ea7fc700 1 --2- 172.21.15.79:0/4187824149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0d3710 0x7fa4ec0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4e9ffb700 1 -- 172.21.15.79:0/4187824149 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971502209 0 0) 0x7fa4e0001110 con 0x7fa4ec0b83c0 2023-06-23T18:36:54.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4e9ffb700 1 -- 172.21.15.79:0/4187824149 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4d40035b0 con 0x7fa4ec0b83c0 2023-06-23T18:36:54.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4e9ffb700 1 -- 172.21.15.79:0/4187824149 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2358312948 0 0) 0x7fa4d40035b0 con 0x7fa4ec0b83c0 2023-06-23T18:36:54.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4e9ffb700 1 -- 172.21.15.79:0/4187824149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0d3710 msgr2=0x7fa4ec0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:54.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4e9ffb700 1 --2- 172.21.15.79:0/4187824149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0d3710 0x7fa4ec0cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4e9ffb700 1 -- 172.21.15.79:0/4187824149 >> v2:172.21.15.155:3300/0 conn(0x7fa4ec0cfa90 msgr2=0x7fa4ec0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:54.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4e9ffb700 1 --2- 172.21.15.79:0/4187824149 >> v2:172.21.15.155:3300/0 conn(0x7fa4ec0cfa90 0x7fa4ec0d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4e9ffb700 1 -- 172.21.15.79:0/4187824149 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec0b8790 con 0x7fa4ec0b83c0 2023-06-23T18:36:54.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4e9ffb700 1 -- 172.21.15.79:0/4187824149 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fa4e0001800 con 0x7fa4ec0b83c0 2023-06-23T18:36:54.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4e9ffb700 1 -- 172.21.15.79:0/4187824149 >> v1:172.21.15.79:6789/0 conn(0x7fa4ec0b83c0 legacy=0x7fa4ec0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4e9ffb700 1 --2- 172.21.15.79:0/4187824149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4d4003a00 0x7fa4d4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.568+0000 7fa4e9ffb700 1 --2- 172.21.15.79:0/4187824149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4d4004430 0x7fa4d40048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4e9ffb700 1 --2- 172.21.15.79:0/4187824149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0d3710 0x7fa4d4005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4e9ffb700 1 -- 172.21.15.79:0/4187824149 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec0cfee0 con 0x7fa4d4003a00 2023-06-23T18:36:54.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4e9ffb700 1 -- 172.21.15.79:0/4187824149 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec0d0a80 con 0x7fa4d4004430 2023-06-23T18:36:54.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4e9ffb700 1 -- 172.21.15.79:0/4187824149 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d400ba10 con 0x7fa4ec0d3710 2023-06-23T18:36:54.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4eaffd700 1 --2- 172.21.15.79:0/4187824149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4d4003a00 0x7fa4d4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4ea7fc700 1 --2- 172.21.15.79:0/4187824149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4d4004430 0x7fa4d40048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4eb7fe700 1 --2- 172.21.15.79:0/4187824149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0d3710 0x7fa4d4005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4eaffd700 1 -- 172.21.15.79:0/4187824149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0d3710 msgr2=0x7fa4d4005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:54.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4eaffd700 1 --2- 172.21.15.79:0/4187824149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0d3710 0x7fa4d4005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4eaffd700 1 -- 172.21.15.79:0/4187824149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4d4004430 msgr2=0x7fa4d40048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:54.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4eaffd700 1 --2- 172.21.15.79:0/4187824149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4d4004430 0x7fa4d40048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4eaffd700 1 -- 172.21.15.79:0/4187824149 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d40035b0 con 0x7fa4d4003a00 2023-06-23T18:36:54.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4ea7fc700 1 --2- 172.21.15.79:0/4187824149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4d4004430 0x7fa4d40048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:54.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4eaffd700 1 --2- 172.21.15.79:0/4187824149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4d4003a00 0x7fa4d4003e10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000c320 tx=0x7fa4d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ec12577ba067d4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:54.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4e9ffb700 1 -- 172.21.15.79:0/4187824149 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e000ce20 con 0x7fa4d4003a00 2023-06-23T18:36:54.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4e9ffb700 1 -- 172.21.15.79:0/4187824149 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e0005960 con 0x7fa4d4003a00 2023-06-23T18:36:54.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.569+0000 7fa4e9ffb700 1 -- 172.21.15.79:0/4187824149 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0005d10 con 0x7fa4d4003a00 2023-06-23T18:36:54.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.570+0000 7fa4f14a7700 1 -- 172.21.15.79:0/4187824149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4d4003a00 msgr2=0x7fa4d4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.570+0000 7fa4f14a7700 1 --2- 172.21.15.79:0/4187824149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4d4003a00 0x7fa4d4003e10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000c320 tx=0x7fa4d4005880 comp rx=0 tx=0).stop 2023-06-23T18:36:54.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.570+0000 7fa4eb7fe700 1 -- 172.21.15.79:0/4187824149 reap_dead start 2023-06-23T18:36:54.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.570+0000 7fa4f14a7700 1 -- 172.21.15.79:0/4187824149 shutdown_connections 2023-06-23T18:36:54.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.570+0000 7fa4f14a7700 1 -- 172.21.15.79:0/4187824149 >> 172.21.15.79:0/4187824149 conn(0x7fa4ec06b350 msgr2=0x7fa4ec06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:54.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.570+0000 7fa4f14a7700 1 -- 172.21.15.79:0/4187824149 shutdown_connections 2023-06-23T18:36:54.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.570+0000 7fa4f14a7700 1 -- 172.21.15.79:0/4187824149 wait complete. 2023-06-23T18:36:54.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.571+0000 7fa4f14a7700 1 Processor -- start 2023-06-23T18:36:54.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.571+0000 7fa4f14a7700 1 -- start start 2023-06-23T18:36:54.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4f14a7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4d4003a00 0x7fa4ec155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4f14a7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4ec153fd0 0x7fa4ec159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4f14a7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4ec159540 0x7fa4ec1567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4f14a7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec1599e0 con 0x7fa4ec159540 2023-06-23T18:36:54.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4f14a7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec154a30 con 0x7fa4ec153fd0 2023-06-23T18:36:54.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4f14a7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec157570 con 0x7fa4d4003a00 2023-06-23T18:36:54.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4eb7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4ec159540 0x7fa4ec1567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4eb7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4ec159540 0x7fa4ec1567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41382/0 (socket says 172.21.15.79:41382) 2023-06-23T18:36:54.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4eb7fe700 1 -- 172.21.15.79:0/813605423 learned_addr learned my addr 172.21.15.79:0/813605423 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:54.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4ea7fc700 1 --2- 172.21.15.79:0/813605423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4ec153fd0 0x7fa4ec159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4eaffd700 1 --2- 172.21.15.79:0/813605423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4d4003a00 0x7fa4ec155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4eb7fe700 1 -- 172.21.15.79:0/813605423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4d4003a00 msgr2=0x7fa4ec155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4eb7fe700 1 --2- 172.21.15.79:0/813605423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4d4003a00 0x7fa4ec155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4eb7fe700 1 -- 172.21.15.79:0/813605423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4ec153fd0 msgr2=0x7fa4ec159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4eb7fe700 1 --2- 172.21.15.79:0/813605423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4ec153fd0 0x7fa4ec159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4eb7fe700 1 -- 172.21.15.79:0/813605423 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec154c40 con 0x7fa4ec159540 2023-06-23T18:36:54.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4eb7fe700 1 --2- 172.21.15.79:0/813605423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4ec159540 0x7fa4ec1567b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec08bdf0 tx=0x7fa4e40114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:54.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4d37fe700 1 -- 172.21.15.79:0/813605423 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e400e070 con 0x7fa4ec159540 2023-06-23T18:36:54.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4d37fe700 1 -- 172.21.15.79:0/813605423 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e4003cf0 con 0x7fa4ec159540 2023-06-23T18:36:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4d37fe700 1 -- 172.21.15.79:0/813605423 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4003ea0 con 0x7fa4ec159540 2023-06-23T18:36:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4f14a7700 1 -- 172.21.15.79:0/813605423 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ec154f20 con 0x7fa4ec159540 2023-06-23T18:36:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.572+0000 7fa4f14a7700 1 -- 172.21.15.79:0/813605423 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ec154440 con 0x7fa4ec159540 2023-06-23T18:36:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.573+0000 7fa4f14a7700 1 -- 172.21.15.79:0/813605423 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4ec155440 con 0x7fa4ec159540 2023-06-23T18:36:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.574+0000 7fa4d37fe700 1 -- 172.21.15.79:0/813605423 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa4e4011a10 con 0x7fa4ec159540 2023-06-23T18:36:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.574+0000 7fa4d37fe700 1 --2- 172.21.15.79:0/813605423 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa4d806ace0 0x7fa4d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.574+0000 7fa4d37fe700 1 -- 172.21.15.79:0/813605423 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa4e40894c0 con 0x7fa4ec159540 2023-06-23T18:36:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.574+0000 7fa4eaffd700 1 --2- 172.21.15.79:0/813605423 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa4d806ace0 0x7fa4d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.574+0000 7fa4eaffd700 1 --2- 172.21.15.79:0/813605423 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa4d806ace0 0x7fa4d806d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec159bc0 tx=0x7fa4e000a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:54.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.578+0000 7fa4d37fe700 1 -- 172.21.15.79:0/813605423 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa4ec155440 con 0x7fa4ec159540 2023-06-23T18:36:54.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.844+0000 7fa4f14a7700 1 -- 172.21.15.79:0/813605423 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fa4ec1545d0 con 0x7fa4ec159540 2023-06-23T18:36:54.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.847+0000 7fa4d37fe700 1 -- 172.21.15.79:0/813605423 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa4e40548c0 con 0x7fa4ec159540 2023-06-23T18:36:54.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.849+0000 7fa4f14a7700 1 -- 172.21.15.79:0/813605423 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa4d806ace0 msgr2=0x7fa4d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.849+0000 7fa4f14a7700 1 --2- 172.21.15.79:0/813605423 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa4d806ace0 0x7fa4d806d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec159bc0 tx=0x7fa4e000a000 comp rx=0 tx=0).stop 2023-06-23T18:36:54.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.849+0000 7fa4f14a7700 1 -- 172.21.15.79:0/813605423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4ec159540 msgr2=0x7fa4ec1567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.849+0000 7fa4f14a7700 1 --2- 172.21.15.79:0/813605423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4ec159540 0x7fa4ec1567b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec08bdf0 tx=0x7fa4e40114b0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.849+0000 7fa4f14a7700 1 -- 172.21.15.79:0/813605423 shutdown_connections 2023-06-23T18:36:54.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.849+0000 7fa4f14a7700 1 --2- 172.21.15.79:0/813605423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4ec159540 0x7fa4ec1567b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.849+0000 7fa4f14a7700 1 --2- 172.21.15.79:0/813605423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4d4003a00 0x7fa4ec155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.849+0000 7fa4f14a7700 1 --2- 172.21.15.79:0/813605423 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa4d806ace0 0x7fa4d806d190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.849+0000 7fa4f14a7700 1 --2- 172.21.15.79:0/813605423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4ec153fd0 0x7fa4ec159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.849+0000 7fa4f14a7700 1 -- 172.21.15.79:0/813605423 >> 172.21.15.79:0/813605423 conn(0x7fa4ec06b350 msgr2=0x7fa4ec15fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:54.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.849+0000 7fa4f14a7700 1 -- 172.21.15.79:0/813605423 shutdown_connections 2023-06-23T18:36:54.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.849+0000 7fa4f14a7700 1 -- 172.21.15.79:0/813605423 wait complete. 2023-06-23T18:36:54.867 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-06-23T18:36:54.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.985+0000 7fd2620ab700 1 Processor -- start 2023-06-23T18:36:54.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.985+0000 7fd2620ab700 1 -- start start 2023-06-23T18:36:54.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.985+0000 7fd2620ab700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd25c071490 0x7fd25c071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.985+0000 7fd2620ab700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd25c0aee30 0x7fd25c071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:54.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd2620ab700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25c1608f0 con 0x7fd25c0b1270 2023-06-23T18:36:54.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd2620ab700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25c0722e0 con 0x7fd25c0aee30 2023-06-23T18:36:54.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd2620ab700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c160e20 con 0x7fd25c071490 2023-06-23T18:36:54.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd25bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd25c0aee30 0x7fd25c071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd25bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd25c0aee30 0x7fd25c071da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41390/0 (socket says 172.21.15.79:41390) 2023-06-23T18:36:54.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd25affd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd25c0b1270 0x7fd25c069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44430/0 (socket says 172.21.15.79:44430) 2023-06-23T18:36:54.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd25bfff700 1 -- 172.21.15.79:0/1076692941 learned_addr learned my addr 172.21.15.79:0/1076692941 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:54.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd25b7fe700 1 --2- 172.21.15.79:0/1076692941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd25c071490 0x7fd25c071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:54.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd25bfff700 1 -- 172.21.15.79:0/1076692941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd25c071490 msgr2=0x7fd25c071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:54.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd25bfff700 1 --2- 172.21.15.79:0/1076692941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd25c071490 0x7fd25c071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd25bfff700 1 -- 172.21.15.79:0/1076692941 >> v1:172.21.15.79:6789/0 conn(0x7fd25c0b1270 legacy=0x7fd25c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd25a7fc700 1 -- 172.21.15.79:0/1076692941 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3939951761 0 0) 0x7fd25c1608f0 con 0x7fd25c0b1270 2023-06-23T18:36:54.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd25bfff700 1 -- 172.21.15.79:0/1076692941 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c0cd4f0 con 0x7fd25c0aee30 2023-06-23T18:36:54.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd25bfff700 1 --2- 172.21.15.79:0/1076692941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd25c0aee30 0x7fd25c071da0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd254009e50 tx=0x7fd25400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3c5596a73fd400 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:54.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd25a7fc700 1 -- 172.21.15.79:0/1076692941 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25400e070 con 0x7fd25c0aee30 2023-06-23T18:36:54.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd25a7fc700 1 -- 172.21.15.79:0/1076692941 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25400b9c0 con 0x7fd25c0aee30 2023-06-23T18:36:54.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.986+0000 7fd25a7fc700 1 -- 172.21.15.79:0/1076692941 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25400bd70 con 0x7fd25c0aee30 2023-06-23T18:36:54.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.987+0000 7fd2620ab700 1 -- 172.21.15.79:0/1076692941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd25c0aee30 msgr2=0x7fd25c071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:54.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.987+0000 7fd2620ab700 1 --2- 172.21.15.79:0/1076692941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd25c0aee30 0x7fd25c071da0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd254009e50 tx=0x7fd25400b330 comp rx=0 tx=0).stop 2023-06-23T18:36:54.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.987+0000 7fd2620ab700 1 -- 172.21.15.79:0/1076692941 shutdown_connections 2023-06-23T18:36:54.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.987+0000 7fd2620ab700 1 --2- 172.21.15.79:0/1076692941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd25c0aee30 0x7fd25c071da0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.987+0000 7fd2620ab700 1 --2- 172.21.15.79:0/1076692941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd25c071490 0x7fd25c071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:54.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.987+0000 7fd2620ab700 1 -- 172.21.15.79:0/1076692941 >> 172.21.15.79:0/1076692941 conn(0x7fd25c06d660 msgr2=0x7fd25c06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:54.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.987+0000 7fd2620ab700 1 -- 172.21.15.79:0/1076692941 shutdown_connections 2023-06-23T18:36:54.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.987+0000 7fd2620ab700 1 -- 172.21.15.79:0/1076692941 wait complete. 2023-06-23T18:36:55.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.988+0000 7fd2620ab700 1 Processor -- start 2023-06-23T18:36:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.989+0000 7fd2620ab700 1 -- start start 2023-06-23T18:36:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.989+0000 7fd2620ab700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd25c071490 0x7fd25c0c34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.989+0000 7fd2620ab700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd25c0aee30 0x7fd25c0c2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd2620ab700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd25c0b1270 0x7fd25c0c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd2620ab700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0c87a0 con 0x7fd25c0b1270 2023-06-23T18:36:55.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd2620ab700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0c3a10 con 0x7fd25c0aee30 2023-06-23T18:36:55.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd2620ab700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c1f6b50 con 0x7fd25c071490 2023-06-23T18:36:55.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd25bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd25c0b1270 0x7fd25c0c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:55.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd25bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd25c0b1270 0x7fd25c0c3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41396/0 (socket says 172.21.15.79:41396) 2023-06-23T18:36:55.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd25bfff700 1 -- 172.21.15.79:0/1400630151 learned_addr learned my addr 172.21.15.79:0/1400630151 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:55.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd25b7fe700 1 --2- 172.21.15.79:0/1400630151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd25c071490 0x7fd25c0c34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:55.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd25affd700 1 --2- 172.21.15.79:0/1400630151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd25c0aee30 0x7fd25c0c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:55.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd25bfff700 1 -- 172.21.15.79:0/1400630151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd25c071490 msgr2=0x7fd25c0c34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:55.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd25bfff700 1 --2- 172.21.15.79:0/1400630151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd25c071490 0x7fd25c0c34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd25bfff700 1 -- 172.21.15.79:0/1400630151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd25c0aee30 msgr2=0x7fd25c0c2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:55.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd25bfff700 1 --2- 172.21.15.79:0/1400630151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd25c0aee30 0x7fd25c0c2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd25bfff700 1 -- 172.21.15.79:0/1400630151 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd254009b30 con 0x7fd25c0b1270 2023-06-23T18:36:55.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd25bfff700 1 --2- 172.21.15.79:0/1400630151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd25c0b1270 0x7fd25c0c3cb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd25400b730 tx=0x7fd254004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:55.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd253fff700 1 -- 172.21.15.79:0/1400630151 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2540099b0 con 0x7fd25c0b1270 2023-06-23T18:36:55.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd253fff700 1 -- 172.21.15.79:0/1400630151 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2540092e0 con 0x7fd25c0b1270 2023-06-23T18:36:55.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd2620ab700 1 -- 172.21.15.79:0/1400630151 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c154330 con 0x7fd25c0b1270 2023-06-23T18:36:55.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd253fff700 1 -- 172.21.15.79:0/1400630151 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25401b650 con 0x7fd25c0b1270 2023-06-23T18:36:55.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.990+0000 7fd2620ab700 1 -- 172.21.15.79:0/1400630151 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c0c5e00 con 0x7fd25c0b1270 2023-06-23T18:36:55.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.991+0000 7fd2620ab700 1 -- 172.21.15.79:0/1400630151 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c0c88e0 con 0x7fd25c0b1270 2023-06-23T18:36:55.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.992+0000 7fd253fff700 1 -- 172.21.15.79:0/1400630151 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd25401b7b0 con 0x7fd25c0b1270 2023-06-23T18:36:55.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.992+0000 7fd253fff700 1 --2- 172.21.15.79:0/1400630151 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd23c06ace0 0x7fd23c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.992+0000 7fd253fff700 1 -- 172.21.15.79:0/1400630151 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd25408a740 con 0x7fd25c0b1270 2023-06-23T18:36:55.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.992+0000 7fd25b7fe700 1 --2- 172.21.15.79:0/1400630151 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd23c06ace0 0x7fd23c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:55.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.992+0000 7fd25b7fe700 1 --2- 172.21.15.79:0/1400630151 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd23c06ace0 0x7fd23c06d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd24400b9b0 tx=0x7fd244008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:55.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:54.996+0000 7fd253fff700 1 -- 172.21.15.79:0/1400630151 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd25c0c88e0 con 0x7fd25c0b1270 2023-06-23T18:36:55.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.262+0000 7fd2620ab700 1 -- 172.21.15.79:0/1400630151 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fd25c0c88e0 con 0x7fd25c0b1270 2023-06-23T18:36:55.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.266+0000 7fd253fff700 1 -- 172.21.15.79:0/1400630151 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd25c0c88e0 con 0x7fd25c0b1270 2023-06-23T18:36:55.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.268+0000 7fd2620ab700 1 -- 172.21.15.79:0/1400630151 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd23c06ace0 msgr2=0x7fd23c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:55.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.268+0000 7fd2620ab700 1 --2- 172.21.15.79:0/1400630151 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd23c06ace0 0x7fd23c06d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd24400b9b0 tx=0x7fd244008040 comp rx=0 tx=0).stop 2023-06-23T18:36:55.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.268+0000 7fd2620ab700 1 -- 172.21.15.79:0/1400630151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd25c0b1270 msgr2=0x7fd25c0c3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:55.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.268+0000 7fd2620ab700 1 --2- 172.21.15.79:0/1400630151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd25c0b1270 0x7fd25c0c3cb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd25400b730 tx=0x7fd254004ad0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.268+0000 7fd2620ab700 1 -- 172.21.15.79:0/1400630151 shutdown_connections 2023-06-23T18:36:55.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.268+0000 7fd2620ab700 1 --2- 172.21.15.79:0/1400630151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd25c0b1270 0x7fd25c0c3cb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.268+0000 7fd2620ab700 1 --2- 172.21.15.79:0/1400630151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd25c071490 0x7fd25c0c34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.268+0000 7fd2620ab700 1 --2- 172.21.15.79:0/1400630151 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd23c06ace0 0x7fd23c06d190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.268+0000 7fd2620ab700 1 --2- 172.21.15.79:0/1400630151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd25c0aee30 0x7fd25c0c2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.268+0000 7fd2620ab700 1 -- 172.21.15.79:0/1400630151 >> 172.21.15.79:0/1400630151 conn(0x7fd25c06d660 msgr2=0x7fd25c154ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:55.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.268+0000 7fd2620ab700 1 -- 172.21.15.79:0/1400630151 shutdown_connections 2023-06-23T18:36:55.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.268+0000 7fd2620ab700 1 -- 172.21.15.79:0/1400630151 wait complete. 2023-06-23T18:36:55.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:55.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.401+0000 7fa5e2ec4700 1 Processor -- start 2023-06-23T18:36:55.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.401+0000 7fa5e2ec4700 1 -- start start 2023-06-23T18:36:55.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.401+0000 7fa5e2ec4700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa5dc159170 0x7fa5dc0cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.401+0000 7fa5e2ec4700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa5dc0cd900 0x7fa5dc0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.401+0000 7fa5e2ec4700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5dc15cd50 con 0x7fa5dc0ce620 2023-06-23T18:36:55.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.401+0000 7fa5e2ec4700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5dc0cd1e0 con 0x7fa5dc159170 2023-06-23T18:36:55.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.401+0000 7fa5e2ec4700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5dc0cdd80 con 0x7fa5dc0cd900 2023-06-23T18:36:55.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.402+0000 7fa5e0c60700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa5dc159170 0x7fa5dc0cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:55.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.402+0000 7fa5e0c60700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa5dc159170 0x7fa5dc0cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41400/0 (socket says 172.21.15.79:41400) 2023-06-23T18:36:55.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.402+0000 7fa5e0c60700 1 -- 172.21.15.79:0/4147607801 learned_addr learned my addr 172.21.15.79:0/4147607801 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:55.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.402+0000 7fa5dbfff700 1 --2- 172.21.15.79:0/4147607801 >> v2:172.21.15.155:3300/0 conn(0x7fa5dc0cd900 0x7fa5dc0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:55.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.402+0000 7fa5e0c60700 1 -- 172.21.15.79:0/4147607801 >> v2:172.21.15.155:3300/0 conn(0x7fa5dc0cd900 msgr2=0x7fa5dc0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:55.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.402+0000 7fa5e0c60700 1 --2- 172.21.15.79:0/4147607801 >> v2:172.21.15.155:3300/0 conn(0x7fa5dc0cd900 0x7fa5dc0ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.402+0000 7fa5e0c60700 1 -- 172.21.15.79:0/4147607801 >> v1:172.21.15.155:6789/0 conn(0x7fa5dc0ce620 legacy=0x7fa5dc071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:55.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.402+0000 7fa5e0c60700 1 -- 172.21.15.79:0/4147607801 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc0cea80 con 0x7fa5dc159170 2023-06-23T18:36:55.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.402+0000 7fa5e0c60700 1 --2- 172.21.15.79:0/4147607801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5dc159170 0x7fa5dc0cd3c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc070f20 tx=0x7fa5d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40dbd8a1bf688c4f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:55.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.402+0000 7fa5db7fe700 1 -- 172.21.15.79:0/4147607801 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d000b870 con 0x7fa5dc159170 2023-06-23T18:36:55.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.402+0000 7fa5db7fe700 1 -- 172.21.15.79:0/4147607801 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d000b9d0 con 0x7fa5dc159170 2023-06-23T18:36:55.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.402+0000 7fa5db7fe700 1 -- 172.21.15.79:0/4147607801 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d000bd60 con 0x7fa5dc159170 2023-06-23T18:36:55.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.403+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/4147607801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5dc159170 msgr2=0x7fa5dc0cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:55.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.403+0000 7fa5e2ec4700 1 --2- 172.21.15.79:0/4147607801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5dc159170 0x7fa5dc0cd3c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc070f20 tx=0x7fa5d0009bd0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.403+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/4147607801 shutdown_connections 2023-06-23T18:36:55.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.403+0000 7fa5e2ec4700 1 --2- 172.21.15.79:0/4147607801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5dc159170 0x7fa5dc0cd3c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.403+0000 7fa5e2ec4700 1 --2- 172.21.15.79:0/4147607801 >> v2:172.21.15.155:3300/0 conn(0x7fa5dc0cd900 0x7fa5dc0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.403+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/4147607801 >> 172.21.15.79:0/4147607801 conn(0x7fa5dc06d660 msgr2=0x7fa5dc06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:55.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.403+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/4147607801 shutdown_connections 2023-06-23T18:36:55.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.403+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/4147607801 wait complete. 2023-06-23T18:36:55.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.404+0000 7fa5e2ec4700 1 Processor -- start 2023-06-23T18:36:55.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.404+0000 7fa5e2ec4700 1 -- start start 2023-06-23T18:36:55.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5e2ec4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5dc0cd900 0x7fa5dc0b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5e2ec4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5dc0ce620 0x7fa5dc0b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5e2ec4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5dc159170 0x7fa5dc0c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5e2ec4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0c60d0 con 0x7fa5dc0ce620 2023-06-23T18:36:55.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5e2ec4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc14b810 con 0x7fa5dc0cd900 2023-06-23T18:36:55.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5e2ec4700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0c4370 con 0x7fa5dc159170 2023-06-23T18:36:55.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5dbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5dc0ce620 0x7fa5dc0b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:55.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5dbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5dc0ce620 0x7fa5dc0b3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41408/0 (socket says 172.21.15.79:41408) 2023-06-23T18:36:55.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5dbfff700 1 -- 172.21.15.79:0/1038644029 learned_addr learned my addr 172.21.15.79:0/1038644029 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:55.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5e0c60700 1 --2- 172.21.15.79:0/1038644029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5dc0cd900 0x7fa5dc0b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:55.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5dbfff700 1 -- 172.21.15.79:0/1038644029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5dc159170 msgr2=0x7fa5dc0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:55.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5dbfff700 1 --2- 172.21.15.79:0/1038644029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5dc159170 0x7fa5dc0c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5dbfff700 1 -- 172.21.15.79:0/1038644029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5dc0cd900 msgr2=0x7fa5dc0b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:55.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5dbfff700 1 --2- 172.21.15.79:0/1038644029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5dc0cd900 0x7fa5dc0b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5dbfff700 1 -- 172.21.15.79:0/1038644029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d0009550 con 0x7fa5dc0ce620 2023-06-23T18:36:55.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.405+0000 7fa5dbfff700 1 --2- 172.21.15.79:0/1038644029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5dc0ce620 0x7fa5dc0b3070 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800e810 tx=0x7fa5c800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:55.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.406+0000 7fa5d97fa700 1 -- 172.21.15.79:0/1038644029 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8013610 con 0x7fa5dc0ce620 2023-06-23T18:36:55.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.406+0000 7fa5d97fa700 1 -- 172.21.15.79:0/1038644029 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c8013770 con 0x7fa5dc0ce620 2023-06-23T18:36:55.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.406+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/1038644029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5dc0c5a80 con 0x7fa5dc0ce620 2023-06-23T18:36:55.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.406+0000 7fa5d97fa700 1 -- 172.21.15.79:0/1038644029 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8013b20 con 0x7fa5dc0ce620 2023-06-23T18:36:55.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.406+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/1038644029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5dc0c4650 con 0x7fa5dc0ce620 2023-06-23T18:36:55.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.407+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/1038644029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5dc0c6240 con 0x7fa5dc0ce620 2023-06-23T18:36:55.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.407+0000 7fa5d97fa700 1 -- 172.21.15.79:0/1038644029 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa5c8013c80 con 0x7fa5dc0ce620 2023-06-23T18:36:55.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.408+0000 7fa5d97fa700 1 --2- 172.21.15.79:0/1038644029 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa5c006ace0 0x7fa5c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.408+0000 7fa5e0c60700 1 --2- 172.21.15.79:0/1038644029 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa5c006ace0 0x7fa5c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:55.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.408+0000 7fa5d97fa700 1 -- 172.21.15.79:0/1038644029 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa5c8088de0 con 0x7fa5dc0ce620 2023-06-23T18:36:55.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.408+0000 7fa5e0c60700 1 --2- 172.21.15.79:0/1038644029 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa5c006ace0 0x7fa5c006d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009c10 tx=0x7fa5d0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:55.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.412+0000 7fa5d97fa700 1 -- 172.21.15.79:0/1038644029 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa5c8057c80 con 0x7fa5dc0ce620 2023-06-23T18:36:55.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.601+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/1038644029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5dc15d830 con 0x7fa5dc0ce620 2023-06-23T18:36:55.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.602+0000 7fa5d97fa700 1 -- 172.21.15.79:0/1038644029 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+14593 (secure 0 0 0) 0x7fa5c8057810 con 0x7fa5dc0ce620 2023-06-23T18:36:55.607 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:36:52.324694+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:55.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.603+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/1038644029 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa5c006ace0 msgr2=0x7fa5c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:55.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.603+0000 7fa5e2ec4700 1 --2- 172.21.15.79:0/1038644029 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa5c006ace0 0x7fa5c006d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009c10 tx=0x7fa5d0008030 comp rx=0 tx=0).stop 2023-06-23T18:36:55.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.603+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/1038644029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5dc0ce620 msgr2=0x7fa5dc0b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:55.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.603+0000 7fa5e2ec4700 1 --2- 172.21.15.79:0/1038644029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5dc0ce620 0x7fa5dc0b3070 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800e810 tx=0x7fa5c800ebd0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.604+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/1038644029 shutdown_connections 2023-06-23T18:36:55.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.604+0000 7fa5e2ec4700 1 --2- 172.21.15.79:0/1038644029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5dc159170 0x7fa5dc0c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.604+0000 7fa5e2ec4700 1 --2- 172.21.15.79:0/1038644029 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa5c006ace0 0x7fa5c006d190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.604+0000 7fa5e2ec4700 1 --2- 172.21.15.79:0/1038644029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5dc0cd900 0x7fa5dc0b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.604+0000 7fa5e2ec4700 1 --2- 172.21.15.79:0/1038644029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5dc0ce620 0x7fa5dc0b3070 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.604+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/1038644029 >> 172.21.15.79:0/1038644029 conn(0x7fa5dc06d660 msgr2=0x7fa5dc1522b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:55.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.604+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/1038644029 shutdown_connections 2023-06-23T18:36:55.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.604+0000 7fa5e2ec4700 1 -- 172.21.15.79:0/1038644029 wait complete. 2023-06-23T18:36:55.620 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:36:55.621 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:36:55.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.739+0000 7fbdcd4a5700 1 Processor -- start 2023-06-23T18:36:55.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.739+0000 7fbdcd4a5700 1 -- start start 2023-06-23T18:36:55.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.740+0000 7fbdcd4a5700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbdc80cfd10 0x7fbdc80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.740+0000 7fbdcd4a5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbdc80b1a80 0x7fbdc80d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.740+0000 7fbdcd4a5700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc80d1830 con 0x7fbdc80b81c0 2023-06-23T18:36:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.740+0000 7fbdcd4a5700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc80cf330 con 0x7fbdc80b1a80 2023-06-23T18:36:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.740+0000 7fbdcd4a5700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc80d1270 con 0x7fbdc80cfd10 2023-06-23T18:36:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.740+0000 7fbdc77fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbdc80b1a80 0x7fbdc80d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.740+0000 7fbdc77fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbdc80b1a80 0x7fbdc80d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41418/0 (socket says 172.21.15.79:41418) 2023-06-23T18:36:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.740+0000 7fbdc77fe700 1 -- 172.21.15.79:0/1307928530 learned_addr learned my addr 172.21.15.79:0/1307928530 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.740+0000 7fbdc77fe700 1 -- 172.21.15.79:0/1307928530 >> v2:172.21.15.155:3300/0 conn(0x7fbdc80cfd10 msgr2=0x7fbdc80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:55.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.740+0000 7fbdc77fe700 1 --2- 172.21.15.79:0/1307928530 >> v2:172.21.15.155:3300/0 conn(0x7fbdc80cfd10 0x7fbdc80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.740+0000 7fbdc77fe700 1 -- 172.21.15.79:0/1307928530 >> v1:172.21.15.155:6789/0 conn(0x7fbdc80b81c0 legacy=0x7fbdc80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:55.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.740+0000 7fbdc77fe700 1 -- 172.21.15.79:0/1307928530 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc80d1550 con 0x7fbdc80b1a80 2023-06-23T18:36:55.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.740+0000 7fbdc77fe700 1 --2- 172.21.15.79:0/1307928530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc80b1a80 0x7fbdc80d28c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80d0770 tx=0x7fbdc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78eea51913a291f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:55.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.741+0000 7fbdc5ffb700 1 -- 172.21.15.79:0/1307928530 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc000e070 con 0x7fbdc80b1a80 2023-06-23T18:36:55.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.741+0000 7fbdc5ffb700 1 -- 172.21.15.79:0/1307928530 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc0003a20 con 0x7fbdc80b1a80 2023-06-23T18:36:55.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.741+0000 7fbdc5ffb700 1 -- 172.21.15.79:0/1307928530 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0003dd0 con 0x7fbdc80b1a80 2023-06-23T18:36:55.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.741+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/1307928530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc80b1a80 msgr2=0x7fbdc80d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:55.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.741+0000 7fbdcd4a5700 1 --2- 172.21.15.79:0/1307928530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc80b1a80 0x7fbdc80d28c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80d0770 tx=0x7fbdc000b330 comp rx=0 tx=0).stop 2023-06-23T18:36:55.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.741+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/1307928530 shutdown_connections 2023-06-23T18:36:55.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.741+0000 7fbdcd4a5700 1 --2- 172.21.15.79:0/1307928530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc80b1a80 0x7fbdc80d28c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.741+0000 7fbdcd4a5700 1 --2- 172.21.15.79:0/1307928530 >> v2:172.21.15.155:3300/0 conn(0x7fbdc80cfd10 0x7fbdc80d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.741+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/1307928530 >> 172.21.15.79:0/1307928530 conn(0x7fbdc806b2b0 msgr2=0x7fbdc806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:55.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.741+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/1307928530 shutdown_connections 2023-06-23T18:36:55.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.741+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/1307928530 wait complete. 2023-06-23T18:36:55.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.742+0000 7fbdcd4a5700 1 Processor -- start 2023-06-23T18:36:55.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.742+0000 7fbdcd4a5700 1 -- start start 2023-06-23T18:36:55.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.743+0000 7fbdcd4a5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc80b1a80 0x7fbdc80b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.743+0000 7fbdcd4a5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc80b81c0 0x7fbdc81f1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.743+0000 7fbdcd4a5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdc80cfd10 0x7fbdc81f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.743+0000 7fbdcd4a5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc81f1d90 con 0x7fbdc80b81c0 2023-06-23T18:36:55.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.743+0000 7fbdcd4a5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80c9270 con 0x7fbdc80cfd10 2023-06-23T18:36:55.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.743+0000 7fbdcd4a5700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80c8660 con 0x7fbdc80b1a80 2023-06-23T18:36:55.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.743+0000 7fbdc67fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc80b81c0 0x7fbdc81f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:55.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.743+0000 7fbdc67fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc80b81c0 0x7fbdc81f1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41422/0 (socket says 172.21.15.79:41422) 2023-06-23T18:36:55.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.743+0000 7fbdc67fc700 1 -- 172.21.15.79:0/4261995587 learned_addr learned my addr 172.21.15.79:0/4261995587 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:55.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.744+0000 7fbdc6ffd700 1 --2- 172.21.15.79:0/4261995587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc80b1a80 0x7fbdc80b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:55.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.744+0000 7fbdc77fe700 1 --2- 172.21.15.79:0/4261995587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdc80cfd10 0x7fbdc81f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:55.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.744+0000 7fbdc67fc700 1 -- 172.21.15.79:0/4261995587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc80b1a80 msgr2=0x7fbdc80b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:55.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.744+0000 7fbdc67fc700 1 --2- 172.21.15.79:0/4261995587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc80b1a80 0x7fbdc80b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.744+0000 7fbdc67fc700 1 -- 172.21.15.79:0/4261995587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdc80cfd10 msgr2=0x7fbdc81f1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:55.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.744+0000 7fbdc67fc700 1 --2- 172.21.15.79:0/4261995587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdc80cfd10 0x7fbdc81f1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.744+0000 7fbdc67fc700 1 -- 172.21.15.79:0/4261995587 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc0009b30 con 0x7fbdc80b81c0 2023-06-23T18:36:55.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.744+0000 7fbdc67fc700 1 --2- 172.21.15.79:0/4261995587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc80b81c0 0x7fbdc81f1310 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800b450 tx=0x7fbdb800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:55.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.744+0000 7fbdaf7fe700 1 -- 172.21.15.79:0/4261995587 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8011830 con 0x7fbdc80b81c0 2023-06-23T18:36:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.744+0000 7fbdaf7fe700 1 -- 172.21.15.79:0/4261995587 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb8011990 con 0x7fbdc80b81c0 2023-06-23T18:36:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.744+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/4261995587 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc80c8940 con 0x7fbdc80b81c0 2023-06-23T18:36:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.744+0000 7fbdaf7fe700 1 -- 172.21.15.79:0/4261995587 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8011d40 con 0x7fbdc80b81c0 2023-06-23T18:36:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.744+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/4261995587 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc80c8f00 con 0x7fbdc80b81c0 2023-06-23T18:36:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.745+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/4261995587 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc81f1f00 con 0x7fbdc80b81c0 2023-06-23T18:36:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.745+0000 7fbdaf7fe700 1 -- 172.21.15.79:0/4261995587 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbdb801a400 con 0x7fbdc80b81c0 2023-06-23T18:36:55.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.746+0000 7fbdaf7fe700 1 --2- 172.21.15.79:0/4261995587 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbdb406ace0 0x7fbdb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:55.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.746+0000 7fbdaf7fe700 1 -- 172.21.15.79:0/4261995587 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbdb8089140 con 0x7fbdc80b81c0 2023-06-23T18:36:55.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.746+0000 7fbdc6ffd700 1 --2- 172.21.15.79:0/4261995587 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbdb406ace0 0x7fbdb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:55.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.746+0000 7fbdc6ffd700 1 --2- 172.21.15.79:0/4261995587 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbdb406ace0 0x7fbdb406d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8153a80 tx=0x7fbdbc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:55.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.750+0000 7fbdaf7fe700 1 -- 172.21.15.79:0/4261995587 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbdb8054540 con 0x7fbdc80b81c0 2023-06-23T18:36:55.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.935+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/4261995587 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbdc804e1d0 con 0x7fbdc80b81c0 2023-06-23T18:36:55.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.936+0000 7fbdaf7fe700 1 -- 172.21.15.79:0/4261995587 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbdb8009280 con 0x7fbdc80b81c0 2023-06-23T18:36:55.941 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:36:55.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.938+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/4261995587 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbdb406ace0 msgr2=0x7fbdb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:55.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.938+0000 7fbdcd4a5700 1 --2- 172.21.15.79:0/4261995587 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbdb406ace0 0x7fbdb406d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8153a80 tx=0x7fbdbc009040 comp rx=0 tx=0).stop 2023-06-23T18:36:55.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.938+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/4261995587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc80b81c0 msgr2=0x7fbdc81f1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:55.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.938+0000 7fbdcd4a5700 1 --2- 172.21.15.79:0/4261995587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc80b81c0 0x7fbdc81f1310 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800b450 tx=0x7fbdb800b810 comp rx=0 tx=0).stop 2023-06-23T18:36:55.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.938+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/4261995587 shutdown_connections 2023-06-23T18:36:55.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.938+0000 7fbdcd4a5700 1 --2- 172.21.15.79:0/4261995587 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbdb406ace0 0x7fbdb406d190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.938+0000 7fbdcd4a5700 1 --2- 172.21.15.79:0/4261995587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdc80cfd10 0x7fbdc81f1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.938+0000 7fbdcd4a5700 1 --2- 172.21.15.79:0/4261995587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc80b1a80 0x7fbdc80b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.938+0000 7fbdcd4a5700 1 --2- 172.21.15.79:0/4261995587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc80b81c0 0x7fbdc81f1310 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:55.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.938+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/4261995587 >> 172.21.15.79:0/4261995587 conn(0x7fbdc806b2b0 msgr2=0x7fbdc80d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:55.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.938+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/4261995587 shutdown_connections 2023-06-23T18:36:55.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:55.938+0000 7fbdcd4a5700 1 -- 172.21.15.79:0/4261995587 wait complete. 2023-06-23T18:36:55.955 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:36:56.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.072+0000 7f26faf84700 1 Processor -- start 2023-06-23T18:36:56.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.072+0000 7f26faf84700 1 -- start start 2023-06-23T18:36:56.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.073+0000 7f26faf84700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f26f4156ae0 0x7f26f415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.073+0000 7f26faf84700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40cc060 0x7f26f40cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.073+0000 7f26faf84700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f40cc9f0 con 0x7f26f4071710 2023-06-23T18:36:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.073+0000 7f26faf84700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f415daf0 con 0x7f26f4156ae0 2023-06-23T18:36:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.073+0000 7f26faf84700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f415a560 con 0x7f26f40cc060 2023-06-23T18:36:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.073+0000 7f26f8d20700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f26f4156ae0 0x7f26f415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.073+0000 7f26f9521700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f26f4071710 0x7f26f4071b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44474/0 (socket says 172.21.15.79:44474) 2023-06-23T18:36:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.073+0000 7f26f9521700 1 -- 172.21.15.79:0/2215909931 learned_addr learned my addr 172.21.15.79:0/2215909931 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.074+0000 7f26f3fff700 1 --2- 172.21.15.79:0/2215909931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40cc060 0x7f26f40cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.074+0000 7f26f8d20700 1 -- 172.21.15.79:0/2215909931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40cc060 msgr2=0x7f26f40cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.074+0000 7f26f8d20700 1 --2- 172.21.15.79:0/2215909931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40cc060 0x7f26f40cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.074+0000 7f26f8d20700 1 -- 172.21.15.79:0/2215909931 >> v1:172.21.15.79:6789/0 conn(0x7f26f4071710 legacy=0x7f26f4071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.074+0000 7f26f8d20700 1 -- 172.21.15.79:0/2215909931 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f4159950 con 0x7f26f4156ae0 2023-06-23T18:36:56.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.074+0000 7f26f37fe700 1 -- 172.21.15.79:0/2215909931 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641557794 0 0) 0x7f26ec001490 con 0x7f26f4071710 2023-06-23T18:36:56.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.074+0000 7f26f3fff700 1 --2- 172.21.15.79:0/2215909931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40cc060 0x7f26f40cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:56.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.074+0000 7f26f8d20700 1 --2- 172.21.15.79:0/2215909931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f4156ae0 0x7f26f415d5b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009840 tx=0x7f26e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702a80202bdb248c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:56.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.074+0000 7f26f37fe700 1 -- 172.21.15.79:0/2215909931 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8011830 con 0x7f26f4156ae0 2023-06-23T18:36:56.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.074+0000 7f26f37fe700 1 -- 172.21.15.79:0/2215909931 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e8011990 con 0x7f26f4156ae0 2023-06-23T18:36:56.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.074+0000 7f26f37fe700 1 -- 172.21.15.79:0/2215909931 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8011d20 con 0x7f26f4156ae0 2023-06-23T18:36:56.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.075+0000 7f26faf84700 1 -- 172.21.15.79:0/2215909931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f4156ae0 msgr2=0x7f26f415d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.075+0000 7f26faf84700 1 --2- 172.21.15.79:0/2215909931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f4156ae0 0x7f26f415d5b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009840 tx=0x7f26e8009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:56.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.075+0000 7f26faf84700 1 -- 172.21.15.79:0/2215909931 shutdown_connections 2023-06-23T18:36:56.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.075+0000 7f26faf84700 1 --2- 172.21.15.79:0/2215909931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f4156ae0 0x7f26f415d5b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.075+0000 7f26faf84700 1 --2- 172.21.15.79:0/2215909931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40cc060 0x7f26f40cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.075+0000 7f26faf84700 1 -- 172.21.15.79:0/2215909931 >> 172.21.15.79:0/2215909931 conn(0x7f26f406afa0 msgr2=0x7f26f406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:56.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.075+0000 7f26faf84700 1 -- 172.21.15.79:0/2215909931 shutdown_connections 2023-06-23T18:36:56.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.075+0000 7f26faf84700 1 -- 172.21.15.79:0/2215909931 wait complete. 2023-06-23T18:36:56.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.076+0000 7f26faf84700 1 Processor -- start 2023-06-23T18:36:56.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.076+0000 7f26faf84700 1 -- start start 2023-06-23T18:36:56.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.077+0000 7f26faf84700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f4071710 0x7f26f40b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.077+0000 7f26faf84700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40cc060 0x7f26f415b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.077+0000 7f26faf84700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f4156ae0 0x7f26f415be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.077+0000 7f26faf84700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f414df30 con 0x7f26f4156ae0 2023-06-23T18:36:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.077+0000 7f26faf84700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40b52f0 con 0x7f26f4071710 2023-06-23T18:36:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.077+0000 7f26faf84700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40b4770 con 0x7f26f40cc060 2023-06-23T18:36:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.077+0000 7f26f9521700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f4156ae0 0x7f26f415be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.077+0000 7f26f9521700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f4156ae0 0x7f26f415be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41438/0 (socket says 172.21.15.79:41438) 2023-06-23T18:36:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26f9521700 1 -- 172.21.15.79:0/121913307 learned_addr learned my addr 172.21.15.79:0/121913307 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26f8d20700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f4071710 0x7f26f40b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26f3fff700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40cc060 0x7f26f415b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26f9521700 1 -- 172.21.15.79:0/121913307 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40cc060 msgr2=0x7f26f415b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26f9521700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40cc060 0x7f26f415b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26f9521700 1 -- 172.21.15.79:0/121913307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f4071710 msgr2=0x7f26f40b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26f9521700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f4071710 0x7f26f40b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26f9521700 1 -- 172.21.15.79:0/121913307 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e80094a0 con 0x7f26f4156ae0 2023-06-23T18:36:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26f8d20700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f4071710 0x7f26f40b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26f3fff700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40cc060 0x7f26f415b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26f9521700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f4156ae0 0x7f26f415be10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00cca0 tx=0x7f26ec00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26f17fa700 1 -- 172.21.15.79:0/121913307 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00cea0 con 0x7f26f4156ae0 2023-06-23T18:36:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26f17fa700 1 -- 172.21.15.79:0/121913307 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec00ada0 con 0x7f26f4156ae0 2023-06-23T18:36:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26faf84700 1 -- 172.21.15.79:0/121913307 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f40b4a50 con 0x7f26f4156ae0 2023-06-23T18:36:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26f17fa700 1 -- 172.21.15.79:0/121913307 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec007310 con 0x7f26f4156ae0 2023-06-23T18:36:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.078+0000 7f26faf84700 1 -- 172.21.15.79:0/121913307 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f40b5010 con 0x7f26f4156ae0 2023-06-23T18:36:56.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.079+0000 7f26faf84700 1 -- 172.21.15.79:0/121913307 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f40cccb0 con 0x7f26f4156ae0 2023-06-23T18:36:56.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.079+0000 7f26f17fa700 1 -- 172.21.15.79:0/121913307 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f26ec007470 con 0x7f26f4156ae0 2023-06-23T18:36:56.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.080+0000 7f26f17fa700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f26d806ace0 0x7f26d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.080+0000 7f26f17fa700 1 -- 172.21.15.79:0/121913307 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f26ec088e20 con 0x7f26f4156ae0 2023-06-23T18:36:56.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.080+0000 7f26f8d20700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f26d806ace0 0x7f26d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.080+0000 7f26f8d20700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f26d806ace0 0x7f26d806d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26e80090b0 tx=0x7f26e801c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:56.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.084+0000 7f26f17fa700 1 -- 172.21.15.79:0/121913307 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f26ec0542b0 con 0x7f26f4156ae0 2023-06-23T18:36:56.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.271+0000 7f26faf84700 1 -- 172.21.15.79:0/121913307 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26f40cc9f0 con 0x7f26f4156ae0 2023-06-23T18:36:56.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.271+0000 7f26f17fa700 1 -- 172.21.15.79:0/121913307 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26f40cc9f0 con 0x7f26f4156ae0 2023-06-23T18:36:56.277 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:36:56.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.273+0000 7f26faf84700 1 -- 172.21.15.79:0/121913307 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f26d806ace0 msgr2=0x7f26d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.273+0000 7f26faf84700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f26d806ace0 0x7f26d806d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26e80090b0 tx=0x7f26e801c040 comp rx=0 tx=0).stop 2023-06-23T18:36:56.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.273+0000 7f26faf84700 1 -- 172.21.15.79:0/121913307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f4156ae0 msgr2=0x7f26f415be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.273+0000 7f26faf84700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f4156ae0 0x7f26f415be10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00cca0 tx=0x7f26ec00a340 comp rx=0 tx=0).stop 2023-06-23T18:36:56.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.273+0000 7f26faf84700 1 -- 172.21.15.79:0/121913307 shutdown_connections 2023-06-23T18:36:56.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.273+0000 7f26faf84700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f4156ae0 0x7f26f415be10 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.273+0000 7f26faf84700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f26d806ace0 0x7f26d806d190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.273+0000 7f26faf84700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f4071710 0x7f26f40b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.273+0000 7f26faf84700 1 --2- 172.21.15.79:0/121913307 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40cc060 0x7f26f415b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.273+0000 7f26faf84700 1 -- 172.21.15.79:0/121913307 >> 172.21.15.79:0/121913307 conn(0x7f26f406afa0 msgr2=0x7f26f40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:56.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.273+0000 7f26faf84700 1 -- 172.21.15.79:0/121913307 shutdown_connections 2023-06-23T18:36:56.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.273+0000 7f26faf84700 1 -- 172.21.15.79:0/121913307 wait complete. 2023-06-23T18:36:56.290 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-23T18:36:56.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.405+0000 7fabd23b7700 1 Processor -- start 2023-06-23T18:36:56.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.406+0000 7fabd23b7700 1 -- start start 2023-06-23T18:36:56.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.406+0000 7fabd23b7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fabcc0cdad0 0x7fabcc0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.406+0000 7fabd23b7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabcc15cd50 con 0x7fabcc159170 2023-06-23T18:36:56.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.406+0000 7fabd23b7700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabcc0ceaa0 con 0x7fabcc0ce620 2023-06-23T18:36:56.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.406+0000 7fabd23b7700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabcc15cfa0 con 0x7fabcc0cdad0 2023-06-23T18:36:56.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.406+0000 7fabcb7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fabcc0cdad0 0x7fabcc0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.406+0000 7fabcbfff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fabcc159170 0x7fabcc0cd3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44484/0 (socket says 172.21.15.79:44484) 2023-06-23T18:36:56.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.407+0000 7fabcb7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fabcc0cdad0 0x7fabcc0ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41446/0 (socket says 172.21.15.79:41446) 2023-06-23T18:36:56.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.407+0000 7fabcbfff700 1 -- 172.21.15.79:0/3265326119 learned_addr learned my addr 172.21.15.79:0/3265326119 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:56.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.407+0000 7fabcb7fe700 1 -- 172.21.15.79:0/3265326119 >> v1:172.21.15.155:6789/0 conn(0x7fabcc0ce620 legacy=0x7fabcc0cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.407+0000 7fabcb7fe700 1 -- 172.21.15.79:0/3265326119 >> v1:172.21.15.79:6789/0 conn(0x7fabcc159170 legacy=0x7fabcc0cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.407+0000 7fabcb7fe700 1 -- 172.21.15.79:0/3265326119 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc15d280 con 0x7fabcc0cdad0 2023-06-23T18:36:56.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.407+0000 7fabcb7fe700 1 --2- 172.21.15.79:0/3265326119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabcc0cdad0 0x7fabcc0ce0e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009840 tx=0x7fabc0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab89a43684d41866 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:56.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.407+0000 7fabcaffd700 1 -- 172.21.15.79:0/3265326119 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0011830 con 0x7fabcc0cdad0 2023-06-23T18:36:56.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.407+0000 7fabcaffd700 1 -- 172.21.15.79:0/3265326119 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc0011990 con 0x7fabcc0cdad0 2023-06-23T18:36:56.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.407+0000 7fabcaffd700 1 -- 172.21.15.79:0/3265326119 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0011d20 con 0x7fabcc0cdad0 2023-06-23T18:36:56.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.408+0000 7fabd23b7700 1 -- 172.21.15.79:0/3265326119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabcc0cdad0 msgr2=0x7fabcc0ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.408+0000 7fabd23b7700 1 --2- 172.21.15.79:0/3265326119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabcc0cdad0 0x7fabcc0ce0e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009840 tx=0x7fabc0009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:56.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.408+0000 7fabd23b7700 1 -- 172.21.15.79:0/3265326119 shutdown_connections 2023-06-23T18:36:56.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.408+0000 7fabd23b7700 1 --2- 172.21.15.79:0/3265326119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabcc0cdad0 0x7fabcc0ce0e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.408+0000 7fabd23b7700 1 -- 172.21.15.79:0/3265326119 >> 172.21.15.79:0/3265326119 conn(0x7fabcc06d660 msgr2=0x7fabcc06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:56.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.408+0000 7fabd23b7700 1 -- 172.21.15.79:0/3265326119 shutdown_connections 2023-06-23T18:36:56.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.408+0000 7fabd23b7700 1 -- 172.21.15.79:0/3265326119 wait complete. 2023-06-23T18:36:56.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.409+0000 7fabd23b7700 1 Processor -- start 2023-06-23T18:36:56.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.409+0000 7fabd23b7700 1 -- start start 2023-06-23T18:36:56.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.410+0000 7fabd23b7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabcc0cdad0 0x7fabcc1e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.410+0000 7fabd23b7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabcc0ce620 0x7fabcc145b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.410+0000 7fabd23b7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabcc159170 0x7fabcc0b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.410+0000 7fabd23b7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc146b30 con 0x7fabcc0cdad0 2023-06-23T18:36:56.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.410+0000 7fabd23b7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc143c90 con 0x7fabcc0ce620 2023-06-23T18:36:56.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.410+0000 7fabd23b7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc146570 con 0x7fabcc159170 2023-06-23T18:36:56.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.410+0000 7fabcbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabcc0cdad0 0x7fabcc1e2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.410+0000 7fabcbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabcc0cdad0 0x7fabcc1e2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41448/0 (socket says 172.21.15.79:41448) 2023-06-23T18:36:56.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.410+0000 7fabcbfff700 1 -- 172.21.15.79:0/2569316271 learned_addr learned my addr 172.21.15.79:0/2569316271 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:56.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.411+0000 7fabcb7fe700 1 --2- 172.21.15.79:0/2569316271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabcc0ce620 0x7fabcc145b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.411+0000 7fabcbfff700 1 -- 172.21.15.79:0/2569316271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabcc159170 msgr2=0x7fabcc0b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.411+0000 7fabd0954700 1 --2- 172.21.15.79:0/2569316271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabcc159170 0x7fabcc0b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.411+0000 7fabcbfff700 1 --2- 172.21.15.79:0/2569316271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabcc159170 0x7fabcc0b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.411+0000 7fabcbfff700 1 -- 172.21.15.79:0/2569316271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabcc0ce620 msgr2=0x7fabcc145b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.411+0000 7fabcbfff700 1 --2- 172.21.15.79:0/2569316271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabcc0ce620 0x7fabcc145b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.411+0000 7fabcbfff700 1 -- 172.21.15.79:0/2569316271 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc00094a0 con 0x7fabcc0cdad0 2023-06-23T18:36:56.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.411+0000 7fabcbfff700 1 --2- 172.21.15.79:0/2569316271 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabcc0cdad0 0x7fabcc1e2e80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00c520 tx=0x7fabbc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:56.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.411+0000 7fabc8ff9700 1 -- 172.21.15.79:0/2569316271 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc005d70 con 0x7fabcc0cdad0 2023-06-23T18:36:56.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.411+0000 7fabc8ff9700 1 -- 172.21.15.79:0/2569316271 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc0070d0 con 0x7fabcc0cdad0 2023-06-23T18:36:56.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.411+0000 7fabd23b7700 1 -- 172.21.15.79:0/2569316271 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabcc146850 con 0x7fabcc0cdad0 2023-06-23T18:36:56.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.411+0000 7fabc8ff9700 1 -- 172.21.15.79:0/2569316271 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc007440 con 0x7fabcc0cdad0 2023-06-23T18:36:56.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.411+0000 7fabd23b7700 1 -- 172.21.15.79:0/2569316271 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabcc1e33c0 con 0x7fabcc0cdad0 2023-06-23T18:36:56.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.412+0000 7fabd23b7700 1 -- 172.21.15.79:0/2569316271 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabcc143af0 con 0x7fabcc0cdad0 2023-06-23T18:36:56.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.412+0000 7fabc8ff9700 1 -- 172.21.15.79:0/2569316271 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fabbc0075a0 con 0x7fabcc0cdad0 2023-06-23T18:36:56.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.413+0000 7fabc8ff9700 1 --2- 172.21.15.79:0/2569316271 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fabb806ad10 0x7fabb806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.413+0000 7fabc8ff9700 1 -- 172.21.15.79:0/2569316271 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fabbc0884c0 con 0x7fabcc0cdad0 2023-06-23T18:36:56.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.413+0000 7fabcb7fe700 1 --2- 172.21.15.79:0/2569316271 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fabb806ad10 0x7fabb806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.413+0000 7fabcb7fe700 1 --2- 172.21.15.79:0/2569316271 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fabb806ad10 0x7fabb806d1c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009840 tx=0x7fabc001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:56.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.417+0000 7fabc8ff9700 1 -- 172.21.15.79:0/2569316271 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fabcc143af0 con 0x7fabcc0cdad0 2023-06-23T18:36:56.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.606+0000 7fabd23b7700 1 -- 172.21.15.79:0/2569316271 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fabcc143af0 con 0x7fabcc0cdad0 2023-06-23T18:36:56.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.607+0000 7fabc8ff9700 1 -- 172.21.15.79:0/2569316271 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+126 (secure 0 0 0) 0x7fabcc143af0 con 0x7fabcc0cdad0 2023-06-23T18:36:56.612 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:56.612 INFO:teuthology.orchestra.run.smithi079.stdout:[ 2023-06-23T18:36:56.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:56.613 INFO:teuthology.orchestra.run.smithi079.stdout: "poolnum": 1, 2023-06-23T18:36:56.613 INFO:teuthology.orchestra.run.smithi079.stdout: "poolname": "rbd" 2023-06-23T18:36:56.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:36:56.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-23T18:36:56.613 INFO:teuthology.orchestra.run.smithi079.stdout: "poolnum": 2, 2023-06-23T18:36:56.613 INFO:teuthology.orchestra.run.smithi079.stdout: "poolname": ".mgr" 2023-06-23T18:36:56.614 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:36:56.614 INFO:teuthology.orchestra.run.smithi079.stdout:] 2023-06-23T18:36:56.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.609+0000 7fabd23b7700 1 -- 172.21.15.79:0/2569316271 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fabb806ad10 msgr2=0x7fabb806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.609+0000 7fabd23b7700 1 --2- 172.21.15.79:0/2569316271 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fabb806ad10 0x7fabb806d1c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009840 tx=0x7fabc001c040 comp rx=0 tx=0).stop 2023-06-23T18:36:56.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.609+0000 7fabd23b7700 1 -- 172.21.15.79:0/2569316271 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabcc0cdad0 msgr2=0x7fabcc1e2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.609+0000 7fabd23b7700 1 --2- 172.21.15.79:0/2569316271 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabcc0cdad0 0x7fabcc1e2e80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00c520 tx=0x7fabbc00c8e0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.609+0000 7fabd23b7700 1 -- 172.21.15.79:0/2569316271 shutdown_connections 2023-06-23T18:36:56.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.609+0000 7fabd23b7700 1 --2- 172.21.15.79:0/2569316271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabcc159170 0x7fabcc0b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.609+0000 7fabd23b7700 1 --2- 172.21.15.79:0/2569316271 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabcc0cdad0 0x7fabcc1e2e80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.609+0000 7fabd23b7700 1 --2- 172.21.15.79:0/2569316271 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fabb806ad10 0x7fabb806d1c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.609+0000 7fabd23b7700 1 --2- 172.21.15.79:0/2569316271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabcc0ce620 0x7fabcc145b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.609+0000 7fabd23b7700 1 -- 172.21.15.79:0/2569316271 >> 172.21.15.79:0/2569316271 conn(0x7fabcc06d660 msgr2=0x7fabcc0b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:56.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.609+0000 7fabd23b7700 1 -- 172.21.15.79:0/2569316271 shutdown_connections 2023-06-23T18:36:56.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.609+0000 7fabd23b7700 1 -- 172.21.15.79:0/2569316271 wait complete. 2023-06-23T18:36:56.626 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-23T18:36:56.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-23T18:36:56.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.746+0000 7fea3cf87700 1 Processor -- start 2023-06-23T18:36:56.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.746+0000 7fea3cf87700 1 -- start start 2023-06-23T18:36:56.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea3cf87700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3815a130 0x7fea3815b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea3cf87700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fea38071780 0x7fea380cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea3cf87700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea380c98d0 con 0x7fea3806cc30 2023-06-23T18:36:56.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea3cf87700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3815b550 con 0x7fea38071780 2023-06-23T18:36:56.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea3cf87700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea38071c80 con 0x7fea3815a130 2023-06-23T18:36:56.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea35d9b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fea38071780 0x7fea380cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea3659c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3815a130 0x7fea3815b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea36d9d700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fea3806cc30 0x7fea3815ded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:52768/0 (socket says 172.21.15.79:52768) 2023-06-23T18:36:56.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea36d9d700 1 -- 172.21.15.79:0/689112715 learned_addr learned my addr 172.21.15.79:0/689112715 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:56.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea3659c700 1 -- 172.21.15.79:0/689112715 >> v2:172.21.15.155:3300/0 conn(0x7fea38071780 msgr2=0x7fea380cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:56.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea3659c700 1 --2- 172.21.15.79:0/689112715 >> v2:172.21.15.155:3300/0 conn(0x7fea38071780 0x7fea380cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea3659c700 1 -- 172.21.15.79:0/689112715 >> v1:172.21.15.155:6789/0 conn(0x7fea3806cc30 legacy=0x7fea3815ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea3659c700 1 -- 172.21.15.79:0/689112715 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3806b250 con 0x7fea3815a130 2023-06-23T18:36:56.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea35d9b700 1 --2- 172.21.15.79:0/689112715 >> v2:172.21.15.155:3300/0 conn(0x7fea38071780 0x7fea380cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:56.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.747+0000 7fea3659c700 1 --2- 172.21.15.79:0/689112715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3815a130 0x7fea3815b010 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009840 tx=0x7fea2c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5270170cb0336a45 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:56.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.748+0000 7fea3559a700 1 -- 172.21.15.79:0/689112715 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c011830 con 0x7fea3815a130 2023-06-23T18:36:56.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.748+0000 7fea3559a700 1 -- 172.21.15.79:0/689112715 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c011990 con 0x7fea3815a130 2023-06-23T18:36:56.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.748+0000 7fea3559a700 1 -- 172.21.15.79:0/689112715 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c011d20 con 0x7fea3815a130 2023-06-23T18:36:56.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.748+0000 7fea3cf87700 1 -- 172.21.15.79:0/689112715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3815a130 msgr2=0x7fea3815b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.748+0000 7fea3cf87700 1 --2- 172.21.15.79:0/689112715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3815a130 0x7fea3815b010 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009840 tx=0x7fea2c009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:56.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.748+0000 7fea3cf87700 1 -- 172.21.15.79:0/689112715 shutdown_connections 2023-06-23T18:36:56.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.748+0000 7fea3cf87700 1 --2- 172.21.15.79:0/689112715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3815a130 0x7fea3815b010 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.748+0000 7fea3cf87700 1 --2- 172.21.15.79:0/689112715 >> v2:172.21.15.155:3300/0 conn(0x7fea38071780 0x7fea380cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.748+0000 7fea3cf87700 1 -- 172.21.15.79:0/689112715 >> 172.21.15.79:0/689112715 conn(0x7fea380cefc0 msgr2=0x7fea380cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:56.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.748+0000 7fea3cf87700 1 -- 172.21.15.79:0/689112715 shutdown_connections 2023-06-23T18:36:56.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.748+0000 7fea3cf87700 1 -- 172.21.15.79:0/689112715 wait complete. 2023-06-23T18:36:56.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.750+0000 7fea3cf87700 1 Processor -- start 2023-06-23T18:36:56.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.750+0000 7fea3cf87700 1 -- start start 2023-06-23T18:36:56.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea3cf87700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea3806cc30 0x7fea380c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea3cf87700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea38071780 0x7fea380c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea3cf87700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3815a130 0x7fea381f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea3cf87700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea38159970 con 0x7fea3806cc30 2023-06-23T18:36:56.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea3cf87700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea380c3ce0 con 0x7fea3815a130 2023-06-23T18:36:56.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea3cf87700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea381f4130 con 0x7fea38071780 2023-06-23T18:36:56.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea3659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea3806cc30 0x7fea380c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea3659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea3806cc30 0x7fea380c2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41460/0 (socket says 172.21.15.79:41460) 2023-06-23T18:36:56.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea3659c700 1 -- 172.21.15.79:0/3200214255 learned_addr learned my addr 172.21.15.79:0/3200214255 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:56.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea35d9b700 1 --2- 172.21.15.79:0/3200214255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea38071780 0x7fea380c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea36d9d700 1 --2- 172.21.15.79:0/3200214255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3815a130 0x7fea381f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea3659c700 1 -- 172.21.15.79:0/3200214255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea38071780 msgr2=0x7fea380c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea3659c700 1 --2- 172.21.15.79:0/3200214255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea38071780 0x7fea380c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea3659c700 1 -- 172.21.15.79:0/3200214255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3815a130 msgr2=0x7fea381f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:56.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea3659c700 1 --2- 172.21.15.79:0/3200214255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3815a130 0x7fea381f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:56.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.751+0000 7fea3659c700 1 -- 172.21.15.79:0/3200214255 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea280094a0 con 0x7fea3806cc30 2023-06-23T18:36:56.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.752+0000 7fea3659c700 1 --2- 172.21.15.79:0/3200214255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea3806cc30 0x7fea380c2af0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009fd0 tx=0x7fea2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:56.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.752+0000 7fea26ffd700 1 -- 172.21.15.79:0/3200214255 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0036a0 con 0x7fea3806cc30 2023-06-23T18:36:56.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.752+0000 7fea26ffd700 1 -- 172.21.15.79:0/3200214255 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c01e070 con 0x7fea3806cc30 2023-06-23T18:36:56.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.752+0000 7fea3cf87700 1 -- 172.21.15.79:0/3200214255 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2c0094a0 con 0x7fea3806cc30 2023-06-23T18:36:56.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.752+0000 7fea26ffd700 1 -- 172.21.15.79:0/3200214255 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c003a40 con 0x7fea3806cc30 2023-06-23T18:36:56.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.752+0000 7fea3cf87700 1 -- 172.21.15.79:0/3200214255 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea380c33b0 con 0x7fea3806cc30 2023-06-23T18:36:56.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.753+0000 7fea3cf87700 1 -- 172.21.15.79:0/3200214255 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea380c3fc0 con 0x7fea3806cc30 2023-06-23T18:36:56.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.753+0000 7fea26ffd700 1 -- 172.21.15.79:0/3200214255 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fea2c003ba0 con 0x7fea3806cc30 2023-06-23T18:36:56.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.754+0000 7fea26ffd700 1 --2- 172.21.15.79:0/3200214255 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fea1806ace0 0x7fea1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:56.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.754+0000 7fea26ffd700 1 -- 172.21.15.79:0/3200214255 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fea2c08a040 con 0x7fea3806cc30 2023-06-23T18:36:56.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.754+0000 7fea35d9b700 1 --2- 172.21.15.79:0/3200214255 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fea1806ace0 0x7fea1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:56.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.754+0000 7fea35d9b700 1 --2- 172.21.15.79:0/3200214255 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fea1806ace0 0x7fea1806d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fea380c9570 tx=0x7fea28009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:56.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.758+0000 7fea26ffd700 1 -- 172.21.15.79:0/3200214255 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fea2c055450 con 0x7fea3806cc30 2023-06-23T18:36:56.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:56.946+0000 7fea3cf87700 1 -- 172.21.15.79:0/3200214255 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fea380672c0 con 0x7fea3806cc30 2023-06-23T18:36:57.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.144+0000 7fea26ffd700 1 -- 172.21.15.79:0/3200214255 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7fea2c0200b0 con 0x7fea3806cc30 2023-06-23T18:36:57.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.147+0000 7fea3cf87700 1 -- 172.21.15.79:0/3200214255 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fea1806ace0 msgr2=0x7fea1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:57.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.147+0000 7fea3cf87700 1 --2- 172.21.15.79:0/3200214255 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fea1806ace0 0x7fea1806d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fea380c9570 tx=0x7fea28009210 comp rx=0 tx=0).stop 2023-06-23T18:36:57.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.147+0000 7fea3cf87700 1 -- 172.21.15.79:0/3200214255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea3806cc30 msgr2=0x7fea380c2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:57.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.147+0000 7fea3cf87700 1 --2- 172.21.15.79:0/3200214255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea3806cc30 0x7fea380c2af0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009fd0 tx=0x7fea2c004750 comp rx=0 tx=0).stop 2023-06-23T18:36:57.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.147+0000 7fea3cf87700 1 -- 172.21.15.79:0/3200214255 shutdown_connections 2023-06-23T18:36:57.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.147+0000 7fea3cf87700 1 --2- 172.21.15.79:0/3200214255 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fea1806ace0 0x7fea1806d190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:57.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.147+0000 7fea3cf87700 1 --2- 172.21.15.79:0/3200214255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3815a130 0x7fea381f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:57.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.147+0000 7fea3cf87700 1 --2- 172.21.15.79:0/3200214255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea3806cc30 0x7fea380c2af0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:57.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.147+0000 7fea3cf87700 1 --2- 172.21.15.79:0/3200214255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea38071780 0x7fea380c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:57.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.147+0000 7fea3cf87700 1 -- 172.21.15.79:0/3200214255 >> 172.21.15.79:0/3200214255 conn(0x7fea380cefc0 msgr2=0x7fea380cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:57.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.147+0000 7fea3cf87700 1 -- 172.21.15.79:0/3200214255 shutdown_connections 2023-06-23T18:36:57.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.147+0000 7fea3cf87700 1 -- 172.21.15.79:0/3200214255 wait complete. 2023-06-23T18:36:57.154 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_metadata' created 2023-06-23T18:36:57.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-23T18:36:57.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.288+0000 7eff93c6f700 1 Processor -- start 2023-06-23T18:36:57.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.289+0000 7eff93c6f700 1 -- start start 2023-06-23T18:36:57.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.289+0000 7eff93c6f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7eff8c0b19d0 0x7eff8c071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:57.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.289+0000 7eff93c6f700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7eff8c0aed90 0x7eff8c0719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:57.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.289+0000 7eff93c6f700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8c15d620 con 0x7eff8c15c710 2023-06-23T18:36:57.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.289+0000 7eff93c6f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8c0ca870 con 0x7eff8c0b19d0 2023-06-23T18:36:57.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.289+0000 7eff93c6f700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8c0cd9d0 con 0x7eff8c0aed90 2023-06-23T18:36:57.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.289+0000 7eff9120a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7eff8c0b19d0 0x7eff8c071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:57.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.290+0000 7eff9120a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7eff8c0b19d0 0x7eff8c071490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41470/0 (socket says 172.21.15.79:41470) 2023-06-23T18:36:57.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.290+0000 7eff9120a700 1 -- 172.21.15.79:0/4279678540 learned_addr learned my addr 172.21.15.79:0/4279678540 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:57.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.290+0000 7eff9220c700 1 --2- 172.21.15.79:0/4279678540 >> v2:172.21.15.155:3300/0 conn(0x7eff8c0aed90 0x7eff8c0719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:57.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.290+0000 7eff9120a700 1 -- 172.21.15.79:0/4279678540 >> v2:172.21.15.155:3300/0 conn(0x7eff8c0aed90 msgr2=0x7eff8c0719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:57.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.290+0000 7eff9120a700 1 --2- 172.21.15.79:0/4279678540 >> v2:172.21.15.155:3300/0 conn(0x7eff8c0aed90 0x7eff8c0719d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:57.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.290+0000 7eff9120a700 1 -- 172.21.15.79:0/4279678540 >> v1:172.21.15.155:6789/0 conn(0x7eff8c15c710 legacy=0x7eff8c0b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:57.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.290+0000 7eff9120a700 1 -- 172.21.15.79:0/4279678540 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c15ccf0 con 0x7eff8c0b19d0 2023-06-23T18:36:57.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.290+0000 7eff9120a700 1 --2- 172.21.15.79:0/4279678540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff8c0b19d0 0x7eff8c071490 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0ce060 tx=0x7eff8800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946513bd1e53cf64 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:57.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.290+0000 7eff90a09700 1 -- 172.21.15.79:0/4279678540 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8800dd30 con 0x7eff8c0b19d0 2023-06-23T18:36:57.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.290+0000 7eff90a09700 1 -- 172.21.15.79:0/4279678540 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8800de90 con 0x7eff8c0b19d0 2023-06-23T18:36:57.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.290+0000 7eff90a09700 1 -- 172.21.15.79:0/4279678540 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff88010690 con 0x7eff8c0b19d0 2023-06-23T18:36:57.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.291+0000 7eff93c6f700 1 -- 172.21.15.79:0/4279678540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff8c0b19d0 msgr2=0x7eff8c071490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:57.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.291+0000 7eff93c6f700 1 --2- 172.21.15.79:0/4279678540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff8c0b19d0 0x7eff8c071490 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0ce060 tx=0x7eff8800be70 comp rx=0 tx=0).stop 2023-06-23T18:36:57.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.291+0000 7eff93c6f700 1 -- 172.21.15.79:0/4279678540 shutdown_connections 2023-06-23T18:36:57.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.291+0000 7eff93c6f700 1 --2- 172.21.15.79:0/4279678540 >> v2:172.21.15.155:3300/0 conn(0x7eff8c0aed90 0x7eff8c0719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:57.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.291+0000 7eff93c6f700 1 --2- 172.21.15.79:0/4279678540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff8c0b19d0 0x7eff8c071490 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:57.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.291+0000 7eff93c6f700 1 -- 172.21.15.79:0/4279678540 >> 172.21.15.79:0/4279678540 conn(0x7eff8c06e200 msgr2=0x7eff8c0cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:57.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.291+0000 7eff93c6f700 1 -- 172.21.15.79:0/4279678540 shutdown_connections 2023-06-23T18:36:57.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.291+0000 7eff93c6f700 1 -- 172.21.15.79:0/4279678540 wait complete. 2023-06-23T18:36:57.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.292+0000 7eff93c6f700 1 Processor -- start 2023-06-23T18:36:57.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.292+0000 7eff93c6f700 1 -- start start 2023-06-23T18:36:57.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff93c6f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff8c0aed90 0x7eff8c0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:57.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff93c6f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff8c0b19d0 0x7eff8c156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:57.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff93c6f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff8c15c710 0x7eff8c1edf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:57.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff93c6f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c15a3c0 con 0x7eff8c0b19d0 2023-06-23T18:36:57.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff93c6f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0b3180 con 0x7eff8c15c710 2023-06-23T18:36:57.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff93c6f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0b3420 con 0x7eff8c0aed90 2023-06-23T18:36:57.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff9120a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff8c0b19d0 0x7eff8c156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:57.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff9120a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff8c0b19d0 0x7eff8c156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41476/0 (socket says 172.21.15.79:41476) 2023-06-23T18:36:57.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff9120a700 1 -- 172.21.15.79:0/3809184090 learned_addr learned my addr 172.21.15.79:0/3809184090 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:57.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff9220c700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff8c15c710 0x7eff8c1edf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:57.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff91a0b700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff8c0aed90 0x7eff8c0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:57.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff9120a700 1 -- 172.21.15.79:0/3809184090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff8c0aed90 msgr2=0x7eff8c0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:57.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff9120a700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff8c0aed90 0x7eff8c0b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:57.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff9120a700 1 -- 172.21.15.79:0/3809184090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff8c15c710 msgr2=0x7eff8c1edf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:57.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff9120a700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff8c15c710 0x7eff8c1edf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:57.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff9120a700 1 -- 172.21.15.79:0/3809184090 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8800b7f0 con 0x7eff8c0b19d0 2023-06-23T18:36:57.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff9220c700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff8c15c710 0x7eff8c1edf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:57.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff9120a700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff8c0b19d0 0x7eff8c156fe0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7eff88000c00 tx=0x7eff88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:57.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff827fc700 1 -- 172.21.15.79:0/3809184090 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8801f070 con 0x7eff8c0b19d0 2023-06-23T18:36:57.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff827fc700 1 -- 172.21.15.79:0/3809184090 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff88003ba0 con 0x7eff8c0b19d0 2023-06-23T18:36:57.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff827fc700 1 -- 172.21.15.79:0/3809184090 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff88010470 con 0x7eff8c0b19d0 2023-06-23T18:36:57.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff93c6f700 1 -- 172.21.15.79:0/3809184090 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8c0b3700 con 0x7eff8c0b19d0 2023-06-23T18:36:57.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.293+0000 7eff93c6f700 1 -- 172.21.15.79:0/3809184090 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8c0b3cc0 con 0x7eff8c0b19d0 2023-06-23T18:36:57.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.294+0000 7eff93c6f700 1 -- 172.21.15.79:0/3809184090 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8c06bf30 con 0x7eff8c0b19d0 2023-06-23T18:36:57.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.295+0000 7eff827fc700 1 -- 172.21.15.79:0/3809184090 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7eff8802c430 con 0x7eff8c0b19d0 2023-06-23T18:36:57.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.295+0000 7eff827fc700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7eff7006ace0 0x7eff7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:57.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.295+0000 7eff827fc700 1 -- 172.21.15.79:0/3809184090 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5580+0+0 (secure 0 0 0) 0x7eff8808c670 con 0x7eff8c0b19d0 2023-06-23T18:36:57.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.296+0000 7eff91a0b700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7eff7006ace0 0x7eff7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:57.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.296+0000 7eff91a0b700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7eff7006ace0 0x7eff7006d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7eff7c000f60 tx=0x7eff7c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:57.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.301+0000 7eff827fc700 1 -- 172.21.15.79:0/3809184090 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7eff88057990 con 0x7eff8c0b19d0 2023-06-23T18:36:57.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:57.489+0000 7eff93c6f700 1 -- 172.21.15.79:0/3809184090 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7eff8c15a500 con 0x7eff8c0b19d0 2023-06-23T18:36:58.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.148+0000 7eff827fc700 1 -- 172.21.15.79:0/3809184090 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7eff8805afb0 con 0x7eff8c0b19d0 2023-06-23T18:36:58.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.153+0000 7eff93c6f700 1 -- 172.21.15.79:0/3809184090 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7eff7006ace0 msgr2=0x7eff7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.153+0000 7eff93c6f700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7eff7006ace0 0x7eff7006d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7eff7c000f60 tx=0x7eff7c009040 comp rx=0 tx=0).stop 2023-06-23T18:36:58.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.153+0000 7eff93c6f700 1 -- 172.21.15.79:0/3809184090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff8c0b19d0 msgr2=0x7eff8c156fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.153+0000 7eff93c6f700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff8c0b19d0 0x7eff8c156fe0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7eff88000c00 tx=0x7eff88004750 comp rx=0 tx=0).stop 2023-06-23T18:36:58.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.153+0000 7eff93c6f700 1 -- 172.21.15.79:0/3809184090 shutdown_connections 2023-06-23T18:36:58.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.153+0000 7eff93c6f700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7eff7006ace0 0x7eff7006d190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.153+0000 7eff93c6f700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff8c15c710 0x7eff8c1edf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.153+0000 7eff93c6f700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff8c0aed90 0x7eff8c0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.153+0000 7eff93c6f700 1 --2- 172.21.15.79:0/3809184090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff8c0b19d0 0x7eff8c156fe0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.153+0000 7eff93c6f700 1 -- 172.21.15.79:0/3809184090 >> 172.21.15.79:0/3809184090 conn(0x7eff8c06e200 msgr2=0x7eff8c155c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:58.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.160+0000 7eff93c6f700 1 -- 172.21.15.79:0/3809184090 shutdown_connections 2023-06-23T18:36:58.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.160+0000 7eff93c6f700 1 -- 172.21.15.79:0/3809184090 wait complete. 2023-06-23T18:36:58.167 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data' created 2023-06-23T18:36:58.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-23T18:36:58.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.302+0000 7f841a8fe700 1 Processor -- start 2023-06-23T18:36:58.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.302+0000 7f841a8fe700 1 -- start start 2023-06-23T18:36:58.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f841a8fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f84140cc060 0x7f84140cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f841a8fe700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84140cc9f0 con 0x7f8414071710 2023-06-23T18:36:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f841a8fe700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841415a560 con 0x7f8414156ae0 2023-06-23T18:36:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f841a8fe700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8414159950 con 0x7f84140cc060 2023-06-23T18:36:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f84137fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f84140cc060 0x7f84140cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f8418e9b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f8414071710 0x7f8414071b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44526/0 (socket says 172.21.15.79:44526) 2023-06-23T18:36:58.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f8418e9b700 1 -- 172.21.15.79:0/1160540964 learned_addr learned my addr 172.21.15.79:0/1160540964 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:58.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f84137fe700 1 -- 172.21.15.79:0/1160540964 >> v1:172.21.15.155:6789/0 conn(0x7f8414156ae0 legacy=0x7f841415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f84137fe700 1 -- 172.21.15.79:0/1160540964 >> v1:172.21.15.79:6789/0 conn(0x7f8414071710 legacy=0x7f8414071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f8412ffd700 1 -- 172.21.15.79:0/1160540964 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231797530 0 0) 0x7f840c001490 con 0x7f8414071710 2023-06-23T18:36:58.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f84137fe700 1 -- 172.21.15.79:0/1160540964 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8414159c30 con 0x7f84140cc060 2023-06-23T18:36:58.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f84137fe700 1 --2- 172.21.15.79:0/1160540964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140cc060 0x7f84140cc4b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8404009840 tx=0x7f8404009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6279f7fae1288b2a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:58.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f8412ffd700 1 -- 172.21.15.79:0/1160540964 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8404011830 con 0x7f84140cc060 2023-06-23T18:36:58.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f8412ffd700 1 -- 172.21.15.79:0/1160540964 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8404011990 con 0x7f84140cc060 2023-06-23T18:36:58.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f8412ffd700 1 -- 172.21.15.79:0/1160540964 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8404011d20 con 0x7f84140cc060 2023-06-23T18:36:58.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f841a8fe700 1 -- 172.21.15.79:0/1160540964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140cc060 msgr2=0x7f84140cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f841a8fe700 1 --2- 172.21.15.79:0/1160540964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140cc060 0x7f84140cc4b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8404009840 tx=0x7f8404009b50 comp rx=0 tx=0).stop 2023-06-23T18:36:58.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f841a8fe700 1 -- 172.21.15.79:0/1160540964 shutdown_connections 2023-06-23T18:36:58.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f841a8fe700 1 --2- 172.21.15.79:0/1160540964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140cc060 0x7f84140cc4b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.303+0000 7f841a8fe700 1 -- 172.21.15.79:0/1160540964 >> 172.21.15.79:0/1160540964 conn(0x7f841406afa0 msgr2=0x7f841406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:58.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.304+0000 7f841a8fe700 1 -- 172.21.15.79:0/1160540964 shutdown_connections 2023-06-23T18:36:58.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.304+0000 7f841a8fe700 1 -- 172.21.15.79:0/1160540964 wait complete. 2023-06-23T18:36:58.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.305+0000 7f841a8fe700 1 Processor -- start 2023-06-23T18:36:58.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.305+0000 7f841a8fe700 1 -- start start 2023-06-23T18:36:58.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.305+0000 7f841a8fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8414071710 0x7f84141525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:58.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f841a8fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140cc060 0x7f84141f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:58.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f841a8fe700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8414156ae0 0x7f84140afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:58.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f841a8fe700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8414154140 con 0x7f84140cc060 2023-06-23T18:36:58.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f841a8fe700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84141f66f0 con 0x7f8414071710 2023-06-23T18:36:58.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f841a8fe700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84141f6130 con 0x7f8414156ae0 2023-06-23T18:36:58.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f84137fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140cc060 0x7f84141f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:58.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f84137fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140cc060 0x7f84141f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41488/0 (socket says 172.21.15.79:41488) 2023-06-23T18:36:58.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f84137fe700 1 -- 172.21.15.79:0/1052530111 learned_addr learned my addr 172.21.15.79:0/1052530111 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:58.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f8418e9b700 1 --2- 172.21.15.79:0/1052530111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8414156ae0 0x7f84140afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:58.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f8413fff700 1 --2- 172.21.15.79:0/1052530111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8414071710 0x7f84141525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:58.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f84137fe700 1 -- 172.21.15.79:0/1052530111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8414156ae0 msgr2=0x7f84140afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f84137fe700 1 --2- 172.21.15.79:0/1052530111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8414156ae0 0x7f84140afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f84137fe700 1 -- 172.21.15.79:0/1052530111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8414071710 msgr2=0x7f84141525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f84137fe700 1 --2- 172.21.15.79:0/1052530111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8414071710 0x7f84141525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f84137fe700 1 -- 172.21.15.79:0/1052530111 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84040094a0 con 0x7f84140cc060 2023-06-23T18:36:58.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f84137fe700 1 --2- 172.21.15.79:0/1052530111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140cc060 0x7f84141f5830 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f84040090b0 tx=0x7f8404004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:58.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f8410ff9700 1 -- 172.21.15.79:0/1052530111 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840401a900 con 0x7f84140cc060 2023-06-23T18:36:58.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f8410ff9700 1 -- 172.21.15.79:0/1052530111 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840401aa60 con 0x7f84140cc060 2023-06-23T18:36:58.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f8410ff9700 1 -- 172.21.15.79:0/1052530111 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840401add0 con 0x7f84140cc060 2023-06-23T18:36:58.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f841a8fe700 1 -- 172.21.15.79:0/1052530111 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84141f6410 con 0x7f84140cc060 2023-06-23T18:36:58.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.306+0000 7f841a8fe700 1 -- 172.21.15.79:0/1052530111 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8414152ae0 con 0x7f84140cc060 2023-06-23T18:36:58.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.307+0000 7f841a8fe700 1 -- 172.21.15.79:0/1052530111 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84140672c0 con 0x7f84140cc060 2023-06-23T18:36:58.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.308+0000 7f8410ff9700 1 -- 172.21.15.79:0/1052530111 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8404022070 con 0x7f84140cc060 2023-06-23T18:36:58.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.308+0000 7f8410ff9700 1 --2- 172.21.15.79:0/1052530111 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f840006ace0 0x7f840006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:58.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.308+0000 7f8413fff700 1 --2- 172.21.15.79:0/1052530111 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f840006ace0 0x7f840006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:58.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.308+0000 7f8410ff9700 1 -- 172.21.15.79:0/1052530111 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f840408a620 con 0x7f84140cc060 2023-06-23T18:36:58.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.309+0000 7f8413fff700 1 --2- 172.21.15.79:0/1052530111 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f840006ace0 0x7f840006d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f84141f5790 tx=0x7f8408009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:58.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.314+0000 7f8410ff9700 1 -- 172.21.15.79:0/1052530111 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8404055990 con 0x7f84140cc060 2023-06-23T18:36:58.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.544+0000 7f841a8fe700 1 -- 172.21.15.79:0/1052530111 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f84140b1be0 con 0x7f84140cc060 2023-06-23T18:36:58.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.550+0000 7f8410ff9700 1 -- 172.21.15.79:0/1052530111 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8404058fb0 con 0x7f84140cc060 2023-06-23T18:36:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.555+0000 7f841a8fe700 1 -- 172.21.15.79:0/1052530111 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f840006ace0 msgr2=0x7f840006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.555+0000 7f841a8fe700 1 --2- 172.21.15.79:0/1052530111 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f840006ace0 0x7f840006d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f84141f5790 tx=0x7f8408009040 comp rx=0 tx=0).stop 2023-06-23T18:36:58.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.555+0000 7f841a8fe700 1 -- 172.21.15.79:0/1052530111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140cc060 msgr2=0x7f84141f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.555+0000 7f841a8fe700 1 --2- 172.21.15.79:0/1052530111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140cc060 0x7f84141f5830 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f84040090b0 tx=0x7f8404004750 comp rx=0 tx=0).stop 2023-06-23T18:36:58.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.556+0000 7f841a8fe700 1 -- 172.21.15.79:0/1052530111 shutdown_connections 2023-06-23T18:36:58.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.556+0000 7f841a8fe700 1 --2- 172.21.15.79:0/1052530111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8414156ae0 0x7f84140afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.556+0000 7f841a8fe700 1 --2- 172.21.15.79:0/1052530111 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f840006ace0 0x7f840006d190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.556+0000 7f841a8fe700 1 --2- 172.21.15.79:0/1052530111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8414071710 0x7f84141525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.556+0000 7f841a8fe700 1 --2- 172.21.15.79:0/1052530111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140cc060 0x7f84141f5830 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.556+0000 7f841a8fe700 1 -- 172.21.15.79:0/1052530111 >> 172.21.15.79:0/1052530111 conn(0x7f841406afa0 msgr2=0x7f841415bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.556+0000 7f841a8fe700 1 -- 172.21.15.79:0/1052530111 shutdown_connections 2023-06-23T18:36:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.557+0000 7f841a8fe700 1 -- 172.21.15.79:0/1052530111 wait complete. 2023-06-23T18:36:58.562 INFO:teuthology.orchestra.run.smithi079.stderr:new fs with metadata pool 5 and data pool 6 2023-06-23T18:36:58.581 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:58.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.697+0000 7f8d0954e700 1 Processor -- start 2023-06-23T18:36:58.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.697+0000 7f8d0954e700 1 -- start start 2023-06-23T18:36:58.704 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d0954e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040b8530 0x7f8d040cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:58.704 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d0954e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d040d0990 con 0x7f8d040d1580 2023-06-23T18:36:58.704 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d0954e700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d040cd9d0 con 0x7f8d040d3580 2023-06-23T18:36:58.704 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d0954e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d040d3af0 con 0x7f8d040b8530 2023-06-23T18:36:58.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d037fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f8d040d1580 0x7f8d0415c8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44556/0 (socket says 172.21.15.79:44556) 2023-06-23T18:36:58.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d037fe700 1 -- 172.21.15.79:0/3407072172 learned_addr learned my addr 172.21.15.79:0/3407072172 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:58.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 -- 172.21.15.79:0/3407072172 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413918673 0 0) 0x7f8d040d0990 con 0x7f8d040d1580 2023-06-23T18:36:58.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d02ffd700 1 --2- 172.21.15.79:0/3407072172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040b8530 0x7f8d040cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:58.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 -- 172.21.15.79:0/3407072172 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cec0035b0 con 0x7f8d040d1580 2023-06-23T18:36:58.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 -- 172.21.15.79:0/3407072172 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1852682223 0 0) 0x7f8cec0035b0 con 0x7f8d040d1580 2023-06-23T18:36:58.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 -- 172.21.15.79:0/3407072172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040b8530 msgr2=0x7f8d040cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:58.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 --2- 172.21.15.79:0/3407072172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040b8530 0x7f8d040cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 -- 172.21.15.79:0/3407072172 >> v1:172.21.15.155:6789/0 conn(0x7f8d040d3580 legacy=0x7f8d040d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 -- 172.21.15.79:0/3407072172 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d040cd410 con 0x7f8d040d1580 2023-06-23T18:36:58.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 -- 172.21.15.79:0/3407072172 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f8cfc0019f0 con 0x7f8d040d1580 2023-06-23T18:36:58.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d02ffd700 1 --2- 172.21.15.79:0/3407072172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040b8530 0x7f8d040cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:58.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 -- 172.21.15.79:0/3407072172 >> v1:172.21.15.79:6789/0 conn(0x7f8d040d1580 legacy=0x7f8d0415c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 --2- 172.21.15.79:0/3407072172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040b8530 0x7f8cec003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:58.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 --2- 172.21.15.79:0/3407072172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cec004020 0x7f8cec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:58.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 --2- 172.21.15.79:0/3407072172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cec004d90 0x7f8cec005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:58.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 -- 172.21.15.79:0/3407072172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d040d3af0 con 0x7f8cec004020 2023-06-23T18:36:58.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 -- 172.21.15.79:0/3407072172 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec00bb20 con 0x7f8cec004d90 2023-06-23T18:36:58.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d01ffb700 1 -- 172.21.15.79:0/3407072172 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec00bce0 con 0x7f8d040b8530 2023-06-23T18:36:58.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.698+0000 7f8d027fc700 1 --2- 172.21.15.79:0/3407072172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cec004020 0x7f8cec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:58.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.699+0000 7f8d02ffd700 1 --2- 172.21.15.79:0/3407072172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040b8530 0x7f8cec003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:58.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.699+0000 7f8d037fe700 1 --2- 172.21.15.79:0/3407072172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cec004d90 0x7f8cec005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:58.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.699+0000 7f8d027fc700 1 -- 172.21.15.79:0/3407072172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040b8530 msgr2=0x7f8cec003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:58.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.699+0000 7f8d027fc700 1 --2- 172.21.15.79:0/3407072172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040b8530 0x7f8cec003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.699+0000 7f8d027fc700 1 -- 172.21.15.79:0/3407072172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cec004d90 msgr2=0x7f8cec005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:58.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.699+0000 7f8d027fc700 1 --2- 172.21.15.79:0/3407072172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cec004d90 0x7f8cec005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.699+0000 7f8d027fc700 1 -- 172.21.15.79:0/3407072172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec0035b0 con 0x7f8cec004020 2023-06-23T18:36:58.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.699+0000 7f8d027fc700 1 --2- 172.21.15.79:0/3407072172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cec004020 0x7f8cec004850 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80018d0 tx=0x7f8cec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561df68439d06ebc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:58.709 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.699+0000 7f8d01ffb700 1 -- 172.21.15.79:0/3407072172 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf800c690 con 0x7f8cec004020 2023-06-23T18:36:58.709 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.699+0000 7f8d01ffb700 1 -- 172.21.15.79:0/3407072172 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf800c7f0 con 0x7f8cec004020 2023-06-23T18:36:58.709 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.699+0000 7f8d01ffb700 1 -- 172.21.15.79:0/3407072172 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf800cba0 con 0x7f8cec004020 2023-06-23T18:36:58.709 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.700+0000 7f8d0954e700 1 -- 172.21.15.79:0/3407072172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cec004020 msgr2=0x7f8cec004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.709 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.700+0000 7f8d0954e700 1 --2- 172.21.15.79:0/3407072172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cec004020 0x7f8cec004850 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80018d0 tx=0x7f8cec005880 comp rx=0 tx=0).stop 2023-06-23T18:36:58.709 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.700+0000 7f8d037fe700 1 -- 172.21.15.79:0/3407072172 reap_dead start 2023-06-23T18:36:58.709 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.700+0000 7f8d0954e700 1 -- 172.21.15.79:0/3407072172 shutdown_connections 2023-06-23T18:36:58.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.700+0000 7f8d0954e700 1 -- 172.21.15.79:0/3407072172 >> 172.21.15.79:0/3407072172 conn(0x7f8d040715c0 msgr2=0x7f8d04071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:58.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.700+0000 7f8d0954e700 1 -- 172.21.15.79:0/3407072172 shutdown_connections 2023-06-23T18:36:58.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.700+0000 7f8d0954e700 1 -- 172.21.15.79:0/3407072172 wait complete. 2023-06-23T18:36:58.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.701+0000 7f8d0954e700 1 Processor -- start 2023-06-23T18:36:58.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.701+0000 7f8d0954e700 1 -- start start 2023-06-23T18:36:58.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d0954e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cec004020 0x7f8d04147550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:58.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d0954e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d040b3920 0x7f8d041f1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:58.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d0954e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040b3d90 0x7f8d041f1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d0954e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d040cb0a0 con 0x7f8d040b3920 2023-06-23T18:36:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d0954e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d04147a90 con 0x7f8cec004020 2023-06-23T18:36:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d0954e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d041f1bc0 con 0x7f8d040b3d90 2023-06-23T18:36:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d027fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d040b3920 0x7f8d041f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d027fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d040b3920 0x7f8d041f1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41524/0 (socket says 172.21.15.79:41524) 2023-06-23T18:36:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d027fc700 1 -- 172.21.15.79:0/378185864 learned_addr learned my addr 172.21.15.79:0/378185864 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:58.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d037fe700 1 --2- 172.21.15.79:0/378185864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040b3d90 0x7f8d041f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:58.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d027fc700 1 -- 172.21.15.79:0/378185864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040b3d90 msgr2=0x7f8d041f1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d02ffd700 1 --2- 172.21.15.79:0/378185864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cec004020 0x7f8d04147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:58.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d027fc700 1 --2- 172.21.15.79:0/378185864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040b3d90 0x7f8d041f1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d027fc700 1 -- 172.21.15.79:0/378185864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cec004020 msgr2=0x7f8d04147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d027fc700 1 --2- 172.21.15.79:0/378185864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cec004020 0x7f8d04147550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.714 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d027fc700 1 -- 172.21.15.79:0/378185864 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d041f6240 con 0x7f8d040b3920 2023-06-23T18:36:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d027fc700 1 --2- 172.21.15.79:0/378185864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d040b3920 0x7f8d041f1140 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800c0e0 tx=0x7f8cf8001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8ceb7fe700 1 -- 172.21.15.79:0/378185864 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf800cb60 con 0x7f8d040b3920 2023-06-23T18:36:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8ceb7fe700 1 -- 172.21.15.79:0/378185864 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf8021070 con 0x7f8d040b3920 2023-06-23T18:36:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d0954e700 1 -- 172.21.15.79:0/378185864 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d041f56c0 con 0x7f8d040b3920 2023-06-23T18:36:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8ceb7fe700 1 -- 172.21.15.79:0/378185864 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8001330 con 0x7f8d040b3920 2023-06-23T18:36:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.702+0000 7f8d0954e700 1 -- 172.21.15.79:0/378185864 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d041f5c80 con 0x7f8d040b3920 2023-06-23T18:36:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.703+0000 7f8d0954e700 1 -- 172.21.15.79:0/378185864 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d04160150 con 0x7f8d040b3920 2023-06-23T18:36:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.704+0000 7f8ceb7fe700 1 -- 172.21.15.79:0/378185864 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8cf8023030 con 0x7f8d040b3920 2023-06-23T18:36:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.704+0000 7f8ceb7fe700 1 --2- 172.21.15.79:0/378185864 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cf006ace0 0x7f8cf006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.704+0000 7f8ceb7fe700 1 -- 172.21.15.79:0/378185864 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8cf808a560 con 0x7f8d040b3920 2023-06-23T18:36:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.704+0000 7f8d02ffd700 1 --2- 172.21.15.79:0/378185864 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cf006ace0 0x7f8cf006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.705+0000 7f8d02ffd700 1 --2- 172.21.15.79:0/378185864 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cf006ace0 0x7f8cf006d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009880 tx=0x7f8cf4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.709+0000 7f8ceb7fe700 1 -- 172.21.15.79:0/378185864 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8cf80557c0 con 0x7f8d040b3920 2023-06-23T18:36:58.898 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.895+0000 7f8d0954e700 1 -- 172.21.15.79:0/378185864 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d0415fe70 con 0x7f8d040b3920 2023-06-23T18:36:58.899 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.896+0000 7f8ceb7fe700 1 -- 172.21.15.79:0/378185864 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+18318 (secure 0 0 0) 0x7f8cf8058de0 con 0x7f8d040b3920 2023-06-23T18:36:58.900 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:58.900 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":33,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:36:58.545193+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:36:56.964534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-23T18:36:57.507393+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:58.902 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.898+0000 7f8d0954e700 1 -- 172.21.15.79:0/378185864 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cf006ace0 msgr2=0x7f8cf006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.902 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.898+0000 7f8d0954e700 1 --2- 172.21.15.79:0/378185864 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cf006ace0 0x7f8cf006d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009880 tx=0x7f8cf4008040 comp rx=0 tx=0).stop 2023-06-23T18:36:58.902 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.898+0000 7f8d0954e700 1 -- 172.21.15.79:0/378185864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d040b3920 msgr2=0x7f8d041f1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:58.902 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.898+0000 7f8d0954e700 1 --2- 172.21.15.79:0/378185864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d040b3920 0x7f8d041f1140 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800c0e0 tx=0x7f8cf8001230 comp rx=0 tx=0).stop 2023-06-23T18:36:58.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.898+0000 7f8d0954e700 1 -- 172.21.15.79:0/378185864 shutdown_connections 2023-06-23T18:36:58.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.898+0000 7f8d0954e700 1 --2- 172.21.15.79:0/378185864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d040b3d90 0x7f8d041f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.898+0000 7f8d0954e700 1 --2- 172.21.15.79:0/378185864 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8cf006ace0 0x7f8cf006d190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.898+0000 7f8d0954e700 1 --2- 172.21.15.79:0/378185864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cec004020 0x7f8d04147550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.898+0000 7f8d0954e700 1 --2- 172.21.15.79:0/378185864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d040b3920 0x7f8d041f1140 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:58.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.898+0000 7f8d0954e700 1 -- 172.21.15.79:0/378185864 >> 172.21.15.79:0/378185864 conn(0x7f8d040715c0 msgr2=0x7f8d04160670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:58.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.898+0000 7f8d0954e700 1 -- 172.21.15.79:0/378185864 shutdown_connections 2023-06-23T18:36:58.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:58.898+0000 7f8d0954e700 1 -- 172.21.15.79:0/378185864 wait complete. 2023-06-23T18:36:58.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:36:58.950 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:36:58.959 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:36:59.037 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.033+0000 7f967f910700 1 Processor -- start 2023-06-23T18:36:59.038 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.034+0000 7f967f910700 1 -- start start 2023-06-23T18:36:59.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.034+0000 7f967f910700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f967806b3e0 0x7f96780cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:59.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.034+0000 7f967f910700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9678071b70 con 0x7f9678069e90 2023-06-23T18:36:59.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.034+0000 7f967f910700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967806d740 con 0x7f9678159fd0 2023-06-23T18:36:59.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.034+0000 7f967f910700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967806d530 con 0x7f967806b3e0 2023-06-23T18:36:59.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f967dead700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9678069e90 0x7f96780b1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44576/0 (socket says 172.21.15.79:44576) 2023-06-23T18:36:59.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f967dead700 1 -- 172.21.15.79:0/3264227942 learned_addr learned my addr 172.21.15.79:0/3264227942 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:59.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f967ceab700 1 --2- 172.21.15.79:0/3264227942 >> v2:172.21.15.155:3300/0 conn(0x7f967806b3e0 0x7f96780cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:59.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 -- 172.21.15.79:0/3264227942 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142107325 0 0) 0x7f9678071b70 con 0x7f9678069e90 2023-06-23T18:36:59.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 -- 172.21.15.79:0/3264227942 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96640035b0 con 0x7f9678069e90 2023-06-23T18:36:59.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 -- 172.21.15.79:0/3264227942 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 562572749 0 0) 0x7f96640035b0 con 0x7f9678069e90 2023-06-23T18:36:59.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 -- 172.21.15.79:0/3264227942 >> v2:172.21.15.155:3300/0 conn(0x7f967806b3e0 msgr2=0x7f96780cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:59.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 --2- 172.21.15.79:0/3264227942 >> v2:172.21.15.155:3300/0 conn(0x7f967806b3e0 0x7f96780cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 -- 172.21.15.79:0/3264227942 >> v1:172.21.15.155:6789/0 conn(0x7f9678159fd0 legacy=0x7f96780c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:59.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 -- 172.21.15.79:0/3264227942 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967806da20 con 0x7f9678069e90 2023-06-23T18:36:59.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 -- 172.21.15.79:0/3264227942 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f96700019f0 con 0x7f9678069e90 2023-06-23T18:36:59.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 -- 172.21.15.79:0/3264227942 >> v1:172.21.15.79:6789/0 conn(0x7f9678069e90 legacy=0x7f96780b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:59.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 --2- 172.21.15.79:0/3264227942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9664003a00 0x7f9664003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:59.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 --2- 172.21.15.79:0/3264227942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9664004850 0x7f9664004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:59.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 --2- 172.21.15.79:0/3264227942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9664005200 0x7f966400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:59.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 -- 172.21.15.79:0/3264227942 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967806d530 con 0x7f9664003a00 2023-06-23T18:36:59.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 -- 172.21.15.79:0/3264227942 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966400c080 con 0x7f9664005200 2023-06-23T18:36:59.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f966ffff700 1 -- 172.21.15.79:0/3264227942 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966400c240 con 0x7f9664004850 2023-06-23T18:36:59.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f967d6ac700 1 --2- 172.21.15.79:0/3264227942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9664003a00 0x7f9664003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:59.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f967dead700 1 --2- 172.21.15.79:0/3264227942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9664005200 0x7f966400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:59.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.035+0000 7f967ceab700 1 --2- 172.21.15.79:0/3264227942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9664004850 0x7f9664004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:59.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f967d6ac700 1 -- 172.21.15.79:0/3264227942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9664004850 msgr2=0x7f9664004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:59.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f967d6ac700 1 --2- 172.21.15.79:0/3264227942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9664004850 0x7f9664004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f967d6ac700 1 -- 172.21.15.79:0/3264227942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9664005200 msgr2=0x7f966400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:59.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f967d6ac700 1 --2- 172.21.15.79:0/3264227942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9664005200 0x7f966400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f967dead700 1 -- 172.21.15.79:0/3264227942 reap_dead start 2023-06-23T18:36:59.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f967d6ac700 1 -- 172.21.15.79:0/3264227942 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96640035b0 con 0x7f9664003a00 2023-06-23T18:36:59.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f967ceab700 1 --2- 172.21.15.79:0/3264227942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9664004850 0x7f9664004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:59.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f967d6ac700 1 --2- 172.21.15.79:0/3264227942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9664003a00 0x7f9664003e10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f96680017c0 tx=0x7f9664005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7a0fce9d60e23e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:59.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f966ffff700 1 -- 172.21.15.79:0/3264227942 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96680052d0 con 0x7f9664003a00 2023-06-23T18:36:59.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f966ffff700 1 -- 172.21.15.79:0/3264227942 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9668005450 con 0x7f9664003a00 2023-06-23T18:36:59.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f966ffff700 1 -- 172.21.15.79:0/3264227942 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96680038c0 con 0x7f9664003a00 2023-06-23T18:36:59.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f967f910700 1 -- 172.21.15.79:0/3264227942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9664003a00 msgr2=0x7f9664003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:59.046 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f967f910700 1 --2- 172.21.15.79:0/3264227942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9664003a00 0x7f9664003e10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f96680017c0 tx=0x7f9664005880 comp rx=0 tx=0).stop 2023-06-23T18:36:59.046 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f967f910700 1 -- 172.21.15.79:0/3264227942 shutdown_connections 2023-06-23T18:36:59.046 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f967f910700 1 --2- 172.21.15.79:0/3264227942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9664003a00 0x7f9664003e10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.046 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f967f910700 1 -- 172.21.15.79:0/3264227942 >> 172.21.15.79:0/3264227942 conn(0x7f96780b17e0 msgr2=0x7f967806cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:59.046 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.036+0000 7f967f910700 1 -- 172.21.15.79:0/3264227942 shutdown_connections 2023-06-23T18:36:59.046 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.037+0000 7f967f910700 1 -- 172.21.15.79:0/3264227942 wait complete. 2023-06-23T18:36:59.048 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.038+0000 7f967f910700 1 Processor -- start 2023-06-23T18:36:59.049 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.038+0000 7f967f910700 1 -- start start 2023-06-23T18:36:59.049 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.038+0000 7f967f910700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9664003a00 0x7f96780c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:59.049 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.038+0000 7f967f910700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f967815b500 0x7f967815b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:59.049 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.038+0000 7f967f910700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96780c7b00 0x7f96780c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:59.049 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.038+0000 7f967f910700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96780b04e0 con 0x7f967815b500 2023-06-23T18:36:59.049 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.038+0000 7f967f910700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96781e44f0 con 0x7f96780c7b00 2023-06-23T18:36:59.050 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.038+0000 7f967f910700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967815b070 con 0x7f9664003a00 2023-06-23T18:36:59.050 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.038+0000 7f967ceab700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f967815b500 0x7f967815b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:59.050 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.038+0000 7f967ceab700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f967815b500 0x7f967815b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41544/0 (socket says 172.21.15.79:41544) 2023-06-23T18:36:59.050 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f967ceab700 1 -- 172.21.15.79:0/1692426844 learned_addr learned my addr 172.21.15.79:0/1692426844 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:59.050 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f967dead700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96780c7b00 0x7f96780c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:59.050 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f967d6ac700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9664003a00 0x7f96780c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:59.051 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f967ceab700 1 -- 172.21.15.79:0/1692426844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9664003a00 msgr2=0x7f96780c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:59.051 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f967ceab700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9664003a00 0x7f96780c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.051 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f967ceab700 1 -- 172.21.15.79:0/1692426844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96780c7b00 msgr2=0x7f96780c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:59.051 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f967ceab700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96780c7b00 0x7f96780c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.051 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f967ceab700 1 -- 172.21.15.79:0/1692426844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967815a740 con 0x7f967815b500 2023-06-23T18:36:59.052 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f967d6ac700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9664003a00 0x7f96780c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:59.052 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f967dead700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96780c7b00 0x7f96780c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:59.052 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f967ceab700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f967815b500 0x7f967815b970 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f96740115c0 tx=0x7f9674011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:59.052 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f966dffb700 1 -- 172.21.15.79:0/1692426844 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674004220 con 0x7f967815b500 2023-06-23T18:36:59.052 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f966dffb700 1 -- 172.21.15.79:0/1692426844 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9674004380 con 0x7f967815b500 2023-06-23T18:36:59.052 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f967f910700 1 -- 172.21.15.79:0/1692426844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967815aa20 con 0x7f967815b500 2023-06-23T18:36:59.052 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f966dffb700 1 -- 172.21.15.79:0/1692426844 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96740036a0 con 0x7f967815b500 2023-06-23T18:36:59.053 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.039+0000 7f967f910700 1 -- 172.21.15.79:0/1692426844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967815beb0 con 0x7f967815b500 2023-06-23T18:36:59.053 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.040+0000 7f967f910700 1 -- 172.21.15.79:0/1692426844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967815b1b0 con 0x7f967815b500 2023-06-23T18:36:59.053 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.041+0000 7f966dffb700 1 -- 172.21.15.79:0/1692426844 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9674003800 con 0x7f967815b500 2023-06-23T18:36:59.053 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.041+0000 7f966dffb700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f965c06ad10 0x7f965c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:59.053 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.041+0000 7f966dffb700 1 -- 172.21.15.79:0/1692426844 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9674089660 con 0x7f967815b500 2023-06-23T18:36:59.054 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.041+0000 7f967d6ac700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f965c06ad10 0x7f965c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:59.054 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.041+0000 7f967d6ac700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f965c06ad10 0x7f965c06d1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9668015040 tx=0x7f9668009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:59.054 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.045+0000 7f966dffb700 1 -- 172.21.15.79:0/1692426844 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f96740547a0 con 0x7f967815b500 2023-06-23T18:36:59.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.237+0000 7f967f910700 1 -- 172.21.15.79:0/1692426844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96780b3f10 con 0x7f967815b500 2023-06-23T18:36:59.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.238+0000 7f966dffb700 1 -- 172.21.15.79:0/1692426844 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+18318 (secure 0 0 0) 0x7f9674057dc0 con 0x7f967815b500 2023-06-23T18:36:59.243 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:36:59.243 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":33,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:36:58.545193+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:36:56.964534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-23T18:36:57.507393+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:36:59.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.240+0000 7f967f910700 1 -- 172.21.15.79:0/1692426844 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f965c06ad10 msgr2=0x7f965c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:59.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.240+0000 7f967f910700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f965c06ad10 0x7f965c06d1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9668015040 tx=0x7f9668009030 comp rx=0 tx=0).stop 2023-06-23T18:36:59.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.240+0000 7f967f910700 1 -- 172.21.15.79:0/1692426844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f967815b500 msgr2=0x7f967815b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:59.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.240+0000 7f967f910700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f967815b500 0x7f967815b970 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f96740115c0 tx=0x7f9674011980 comp rx=0 tx=0).stop 2023-06-23T18:36:59.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.240+0000 7f967f910700 1 -- 172.21.15.79:0/1692426844 shutdown_connections 2023-06-23T18:36:59.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.240+0000 7f967f910700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f965c06ad10 0x7f965c06d1c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.240+0000 7f967f910700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96780c7b00 0x7f96780c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.240+0000 7f967f910700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9664003a00 0x7f96780c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.240+0000 7f967f910700 1 --2- 172.21.15.79:0/1692426844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f967815b500 0x7f967815b970 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.240+0000 7f967f910700 1 -- 172.21.15.79:0/1692426844 >> 172.21.15.79:0/1692426844 conn(0x7f96780b17e0 msgr2=0x7f9678156e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:59.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.241+0000 7f967f910700 1 -- 172.21.15.79:0/1692426844 shutdown_connections 2023-06-23T18:36:59.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.241+0000 7f967f910700 1 -- 172.21.15.79:0/1692426844 wait complete. 2023-06-23T18:36:59.257 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-23T18:36:59.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.417+0000 7fc073861700 1 Processor -- start 2023-06-23T18:36:59.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.417+0000 7fc073861700 1 -- start start 2023-06-23T18:36:59.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc073861700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc06c0c90a0 0x7fc06c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:59.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc073861700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc06c06a3d0 0x7fc06c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:59.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc073861700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06c06a830 con 0x7fc06c15c020 2023-06-23T18:36:59.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc073861700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06c15d400 con 0x7fc06c0c90a0 2023-06-23T18:36:59.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc073861700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c06cf40 con 0x7fc06c06a3d0 2023-06-23T18:36:59.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc070dfc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc06c0c90a0 0x7fc06c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:59.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc0715fd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fc06c15c020 0x7fc06c15c460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44592/0 (socket says 172.21.15.79:44592) 2023-06-23T18:36:59.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc0715fd700 1 -- 172.21.15.79:0/1855313071 learned_addr learned my addr 172.21.15.79:0/1855313071 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:36:59.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc071dfe700 1 --2- 172.21.15.79:0/1855313071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc06c06a3d0 0x7fc06c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:59.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc070dfc700 1 -- 172.21.15.79:0/1855313071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc06c06a3d0 msgr2=0x7fc06c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:59.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc070dfc700 1 --2- 172.21.15.79:0/1855313071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc06c06a3d0 0x7fc06c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc070dfc700 1 -- 172.21.15.79:0/1855313071 >> v1:172.21.15.79:6789/0 conn(0x7fc06c15c020 legacy=0x7fc06c15c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:59.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc063fff700 1 -- 172.21.15.79:0/1855313071 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503078539 0 0) 0x7fc06c06a830 con 0x7fc06c15c020 2023-06-23T18:36:59.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc070dfc700 1 -- 172.21.15.79:0/1855313071 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c15cb70 con 0x7fc06c0c90a0 2023-06-23T18:36:59.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc070dfc700 1 --2- 172.21.15.79:0/1855313071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc06c0c90a0 0x7fc06c069e90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00a9e0 tx=0x7fc05c00acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a8281d2723c9bd server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:59.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc063fff700 1 -- 172.21.15.79:0/1855313071 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c00cb70 con 0x7fc06c0c90a0 2023-06-23T18:36:59.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc063fff700 1 -- 172.21.15.79:0/1855313071 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05c00ccd0 con 0x7fc06c0c90a0 2023-06-23T18:36:59.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.418+0000 7fc063fff700 1 -- 172.21.15.79:0/1855313071 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c018620 con 0x7fc06c0c90a0 2023-06-23T18:36:59.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.419+0000 7fc073861700 1 -- 172.21.15.79:0/1855313071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc06c0c90a0 msgr2=0x7fc06c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:59.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.419+0000 7fc073861700 1 --2- 172.21.15.79:0/1855313071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc06c0c90a0 0x7fc06c069e90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00a9e0 tx=0x7fc05c00acf0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.419+0000 7fc073861700 1 -- 172.21.15.79:0/1855313071 shutdown_connections 2023-06-23T18:36:59.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.419+0000 7fc073861700 1 --2- 172.21.15.79:0/1855313071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc06c06a3d0 0x7fc06c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.419+0000 7fc073861700 1 --2- 172.21.15.79:0/1855313071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc06c0c90a0 0x7fc06c069e90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.419+0000 7fc073861700 1 -- 172.21.15.79:0/1855313071 >> 172.21.15.79:0/1855313071 conn(0x7fc06c0cbf10 msgr2=0x7fc06c0ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:59.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.419+0000 7fc073861700 1 -- 172.21.15.79:0/1855313071 shutdown_connections 2023-06-23T18:36:59.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.419+0000 7fc073861700 1 -- 172.21.15.79:0/1855313071 wait complete. 2023-06-23T18:36:59.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.420+0000 7fc073861700 1 Processor -- start 2023-06-23T18:36:59.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.420+0000 7fc073861700 1 -- start start 2023-06-23T18:36:59.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc073861700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc06c06a3d0 0x7fc06c1e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:59.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc073861700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc06c0c90a0 0x7fc06c14aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:59.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc073861700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc06c15c020 0x7fc06c0b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:59.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc073861700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c0b2a30 con 0x7fc06c15c020 2023-06-23T18:36:59.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc073861700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c1e5280 con 0x7fc06c06a3d0 2023-06-23T18:36:59.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc073861700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c0b1f50 con 0x7fc06c0c90a0 2023-06-23T18:36:59.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc071dfe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc06c15c020 0x7fc06c0b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:59.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc071dfe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc06c15c020 0x7fc06c0b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41558/0 (socket says 172.21.15.79:41558) 2023-06-23T18:36:59.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc071dfe700 1 -- 172.21.15.79:0/589271476 learned_addr learned my addr 172.21.15.79:0/589271476 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:36:59.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc070dfc700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc06c0c90a0 0x7fc06c14aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:59.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc0715fd700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc06c06a3d0 0x7fc06c1e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:59.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc071dfe700 1 -- 172.21.15.79:0/589271476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc06c0c90a0 msgr2=0x7fc06c14aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:59.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc071dfe700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc06c0c90a0 0x7fc06c14aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc071dfe700 1 -- 172.21.15.79:0/589271476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc06c06a3d0 msgr2=0x7fc06c1e4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:59.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc071dfe700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc06c06a3d0 0x7fc06c1e4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:59.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc071dfe700 1 -- 172.21.15.79:0/589271476 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c00a640 con 0x7fc06c15c020 2023-06-23T18:36:59.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc070dfc700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc06c0c90a0 0x7fc06c14aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:59.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.421+0000 7fc0715fd700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc06c06a3d0 0x7fc06c1e4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:36:59.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.422+0000 7fc071dfe700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc06c15c020 0x7fc06c0b2e50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc064007e90 tx=0x7fc06400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:59.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.422+0000 7fc061ffb700 1 -- 172.21.15.79:0/589271476 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06400ede0 con 0x7fc06c15c020 2023-06-23T18:36:59.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.422+0000 7fc061ffb700 1 -- 172.21.15.79:0/589271476 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc064014420 con 0x7fc06c15c020 2023-06-23T18:36:59.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.422+0000 7fc073861700 1 -- 172.21.15.79:0/589271476 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06c0b1c70 con 0x7fc06c15c020 2023-06-23T18:36:59.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.422+0000 7fc061ffb700 1 -- 172.21.15.79:0/589271476 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0640147d0 con 0x7fc06c15c020 2023-06-23T18:36:59.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.422+0000 7fc073861700 1 -- 172.21.15.79:0/589271476 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06c0b2650 con 0x7fc06c15c020 2023-06-23T18:36:59.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.423+0000 7fc073861700 1 -- 172.21.15.79:0/589271476 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06c0b2b70 con 0x7fc06c15c020 2023-06-23T18:36:59.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.424+0000 7fc061ffb700 1 -- 172.21.15.79:0/589271476 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc064014930 con 0x7fc06c15c020 2023-06-23T18:36:59.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.424+0000 7fc061ffb700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc05006ace0 0x7fc05006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:59.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.424+0000 7fc061ffb700 1 -- 172.21.15.79:0/589271476 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc064089c40 con 0x7fc06c15c020 2023-06-23T18:36:59.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.424+0000 7fc0715fd700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc05006ace0 0x7fc05006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:59.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.424+0000 7fc0715fd700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc05006ace0 0x7fc05006d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc068000f60 tx=0x7fc068009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:59.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.428+0000 7fc061ffb700 1 -- 172.21.15.79:0/589271476 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc06c0b2b70 con 0x7fc06c15c020 2023-06-23T18:36:59.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:36:59.662+0000 7fc073861700 1 -- 172.21.15.79:0/589271476 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc06c0b2b70 con 0x7fc06c15c020 2023-06-23T18:37:00.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.559+0000 7fc061ffb700 1 -- 172.21.15.79:0/589271476 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc06c0b2b70 con 0x7fc06c15c020 2023-06-23T18:37:00.566 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.562+0000 7fc073861700 1 -- 172.21.15.79:0/589271476 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc05006ace0 msgr2=0x7fc05006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:00.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.562+0000 7fc073861700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc05006ace0 0x7fc05006d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc068000f60 tx=0x7fc068009040 comp rx=0 tx=0).stop 2023-06-23T18:37:00.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.562+0000 7fc073861700 1 -- 172.21.15.79:0/589271476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc06c15c020 msgr2=0x7fc06c0b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:00.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.562+0000 7fc073861700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc06c15c020 0x7fc06c0b2e50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc064007e90 tx=0x7fc06400e3c0 comp rx=0 tx=0).stop 2023-06-23T18:37:00.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.562+0000 7fc073861700 1 -- 172.21.15.79:0/589271476 shutdown_connections 2023-06-23T18:37:00.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.562+0000 7fc073861700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc06c15c020 0x7fc06c0b2e50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:00.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.562+0000 7fc073861700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc05006ace0 0x7fc05006d190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:00.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.562+0000 7fc073861700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc06c06a3d0 0x7fc06c1e4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:00.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.562+0000 7fc073861700 1 --2- 172.21.15.79:0/589271476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc06c0c90a0 0x7fc06c14aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:00.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.562+0000 7fc073861700 1 -- 172.21.15.79:0/589271476 >> 172.21.15.79:0/589271476 conn(0x7fc06c0cbf10 msgr2=0x7fc06c0c99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:00.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.563+0000 7fc073861700 1 -- 172.21.15.79:0/589271476 shutdown_connections 2023-06-23T18:37:00.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.563+0000 7fc073861700 1 -- 172.21.15.79:0/589271476 wait complete. 2023-06-23T18:37:00.582 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-06-23T18:37:00.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-06-23T18:37:00.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.706+0000 7f9dffd30700 1 Processor -- start 2023-06-23T18:37:00.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.706+0000 7f9dffd30700 1 -- start start 2023-06-23T18:37:00.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.707+0000 7f9dffd30700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9df80cc060 0x7f9df80cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:00.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.707+0000 7f9dffd30700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df80cc9f0 con 0x7f9df8156ae0 2023-06-23T18:37:00.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.707+0000 7f9dffd30700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df815a560 con 0x7f9df8071710 2023-06-23T18:37:00.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.707+0000 7f9dffd30700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df8159950 con 0x7f9df80cc060 2023-06-23T18:37:00.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.707+0000 7f9dfd2cb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9df80cc060 0x7f9df80cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:00.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.707+0000 7f9dfdacc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9df8156ae0 0x7f9df815d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44604/0 (socket says 172.21.15.79:44604) 2023-06-23T18:37:00.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.707+0000 7f9dfdacc700 1 -- 172.21.15.79:0/3006900113 learned_addr learned my addr 172.21.15.79:0/3006900113 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:00.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.707+0000 7f9dfd2cb700 1 -- 172.21.15.79:0/3006900113 >> v1:172.21.15.155:6789/0 conn(0x7f9df8071710 legacy=0x7f9df8071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:00.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.707+0000 7f9dfd2cb700 1 -- 172.21.15.79:0/3006900113 >> v1:172.21.15.79:6789/0 conn(0x7f9df8156ae0 legacy=0x7f9df815d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:00.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.707+0000 7f9dfd2cb700 1 -- 172.21.15.79:0/3006900113 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df8159c30 con 0x7f9df80cc060 2023-06-23T18:37:00.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.707+0000 7f9dfcaca700 1 -- 172.21.15.79:0/3006900113 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999920080 0 0) 0x7f9df4001110 con 0x7f9df8156ae0 2023-06-23T18:37:00.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.707+0000 7f9dfd2cb700 1 --2- 172.21.15.79:0/3006900113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df80cc060 0x7f9df80cc4b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009840 tx=0x7f9de8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5de9af55ba7e7be server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:00.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.708+0000 7f9dfcaca700 1 -- 172.21.15.79:0/3006900113 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8004230 con 0x7f9df80cc060 2023-06-23T18:37:00.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.708+0000 7f9dfcaca700 1 -- 172.21.15.79:0/3006900113 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de8004390 con 0x7f9df80cc060 2023-06-23T18:37:00.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.708+0000 7f9dfcaca700 1 -- 172.21.15.79:0/3006900113 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de80036a0 con 0x7f9df80cc060 2023-06-23T18:37:00.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.708+0000 7f9dffd30700 1 -- 172.21.15.79:0/3006900113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df80cc060 msgr2=0x7f9df80cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:00.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.708+0000 7f9dffd30700 1 --2- 172.21.15.79:0/3006900113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df80cc060 0x7f9df80cc4b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009840 tx=0x7f9de8009b50 comp rx=0 tx=0).stop 2023-06-23T18:37:00.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.708+0000 7f9dffd30700 1 -- 172.21.15.79:0/3006900113 shutdown_connections 2023-06-23T18:37:00.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.708+0000 7f9dffd30700 1 --2- 172.21.15.79:0/3006900113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df80cc060 0x7f9df80cc4b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:00.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.708+0000 7f9dffd30700 1 -- 172.21.15.79:0/3006900113 >> 172.21.15.79:0/3006900113 conn(0x7f9df806afa0 msgr2=0x7f9df806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:00.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.708+0000 7f9dffd30700 1 -- 172.21.15.79:0/3006900113 shutdown_connections 2023-06-23T18:37:00.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.708+0000 7f9dffd30700 1 -- 172.21.15.79:0/3006900113 wait complete. 2023-06-23T18:37:00.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.709+0000 7f9dffd30700 1 Processor -- start 2023-06-23T18:37:00.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.709+0000 7f9dffd30700 1 -- start start 2023-06-23T18:37:00.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.710+0000 7f9dffd30700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9df8071710 0x7f9df81487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:00.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.710+0000 7f9dffd30700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9df80cc060 0x7f9df80ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:00.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.710+0000 7f9dffd30700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df8156ae0 0x7f9df8145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:00.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.710+0000 7f9dffd30700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df81593e0 con 0x7f9df8156ae0 2023-06-23T18:37:00.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.710+0000 7f9dffd30700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80abdf0 con 0x7f9df80cc060 2023-06-23T18:37:00.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.710+0000 7f9dffd30700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df81458c0 con 0x7f9df8071710 2023-06-23T18:37:00.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.710+0000 7f9dfe2cd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df8156ae0 0x7f9df8145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:00.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.710+0000 7f9dfe2cd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df8156ae0 0x7f9df8145380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41572/0 (socket says 172.21.15.79:41572) 2023-06-23T18:37:00.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.710+0000 7f9dfe2cd700 1 -- 172.21.15.79:0/3076729450 learned_addr learned my addr 172.21.15.79:0/3076729450 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:00.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.710+0000 7f9dfd2cb700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9df80cc060 0x7f9df80ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:00.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.711+0000 7f9dfdacc700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9df8071710 0x7f9df81487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:00.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.711+0000 7f9dfe2cd700 1 -- 172.21.15.79:0/3076729450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9df8071710 msgr2=0x7f9df81487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:00.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.711+0000 7f9dfe2cd700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9df8071710 0x7f9df81487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:00.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.711+0000 7f9dfe2cd700 1 -- 172.21.15.79:0/3076729450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9df80cc060 msgr2=0x7f9df80ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:00.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.711+0000 7f9dfe2cd700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9df80cc060 0x7f9df80ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:00.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.711+0000 7f9dfe2cd700 1 -- 172.21.15.79:0/3076729450 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de80094a0 con 0x7f9df8156ae0 2023-06-23T18:37:00.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.711+0000 7f9dfdacc700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9df8071710 0x7f9df81487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:00.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.711+0000 7f9dfe2cd700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df8156ae0 0x7f9df8145380 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9df000cc40 tx=0x7f9df000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:00.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.711+0000 7f9dee7fc700 1 -- 172.21.15.79:0/3076729450 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df000aba0 con 0x7f9df8156ae0 2023-06-23T18:37:00.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.711+0000 7f9dee7fc700 1 -- 172.21.15.79:0/3076729450 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df000ad00 con 0x7f9df8156ae0 2023-06-23T18:37:00.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.711+0000 7f9dffd30700 1 -- 172.21.15.79:0/3076729450 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df80ac020 con 0x7f9df8156ae0 2023-06-23T18:37:00.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.711+0000 7f9dee7fc700 1 -- 172.21.15.79:0/3076729450 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0012660 con 0x7f9df8156ae0 2023-06-23T18:37:00.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.711+0000 7f9dffd30700 1 -- 172.21.15.79:0/3076729450 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df8147d70 con 0x7f9df8156ae0 2023-06-23T18:37:00.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.712+0000 7f9dffd30700 1 -- 172.21.15.79:0/3076729450 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df80672c0 con 0x7f9df8156ae0 2023-06-23T18:37:00.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.712+0000 7f9dee7fc700 1 -- 172.21.15.79:0/3076729450 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9df00127c0 con 0x7f9df8156ae0 2023-06-23T18:37:00.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.713+0000 7f9dee7fc700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9ddc06ace0 0x7f9ddc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:00.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.713+0000 7f9dfdacc700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9ddc06ace0 0x7f9ddc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:00.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.713+0000 7f9dee7fc700 1 -- 172.21.15.79:0/3076729450 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9df0089b90 con 0x7f9df8156ae0 2023-06-23T18:37:00.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.713+0000 7f9dfdacc700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9ddc06ace0 0x7f9ddc06d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9df4000f60 tx=0x7f9df4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:00.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.717+0000 7f9dee7fc700 1 -- 172.21.15.79:0/3076729450 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9df0054cd0 con 0x7f9df8156ae0 2023-06-23T18:37:00.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:00.946+0000 7f9dffd30700 1 -- 172.21.15.79:0/3076729450 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9df80b1be0 con 0x7f9df8156ae0 2023-06-23T18:37:01.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.562+0000 7f9dee7fc700 1 -- 172.21.15.79:0/3076729450 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9df0016020 con 0x7f9df8156ae0 2023-06-23T18:37:01.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.565+0000 7f9dffd30700 1 -- 172.21.15.79:0/3076729450 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9ddc06ace0 msgr2=0x7f9ddc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:01.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.565+0000 7f9dffd30700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9ddc06ace0 0x7f9ddc06d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9df4000f60 tx=0x7f9df4009040 comp rx=0 tx=0).stop 2023-06-23T18:37:01.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.565+0000 7f9dffd30700 1 -- 172.21.15.79:0/3076729450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df8156ae0 msgr2=0x7f9df8145380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:01.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.565+0000 7f9dffd30700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df8156ae0 0x7f9df8145380 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9df000cc40 tx=0x7f9df000a060 comp rx=0 tx=0).stop 2023-06-23T18:37:01.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.565+0000 7f9dffd30700 1 -- 172.21.15.79:0/3076729450 shutdown_connections 2023-06-23T18:37:01.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.565+0000 7f9dffd30700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df8156ae0 0x7f9df8145380 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:01.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.565+0000 7f9dffd30700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9df8071710 0x7f9df81487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:01.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.565+0000 7f9dffd30700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9ddc06ace0 0x7f9ddc06d190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:01.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.565+0000 7f9dffd30700 1 --2- 172.21.15.79:0/3076729450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9df80cc060 0x7f9df80ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:01.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.565+0000 7f9dffd30700 1 -- 172.21.15.79:0/3076729450 >> 172.21.15.79:0/3076729450 conn(0x7f9df806afa0 msgr2=0x7f9df80ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:01.572 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.565+0000 7f9dffd30700 1 -- 172.21.15.79:0/3076729450 shutdown_connections 2023-06-23T18:37:01.572 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.565+0000 7f9dffd30700 1 -- 172.21.15.79:0/3076729450 wait complete. 2023-06-23T18:37:01.583 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-06-23T18:37:01.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:37:01.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.703+0000 7f3666aae700 1 Processor -- start 2023-06-23T18:37:01.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.703+0000 7f3666aae700 1 -- start start 2023-06-23T18:37:01.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.703+0000 7f3666aae700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f36600b83c0 0x7f36600cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:01.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.703+0000 7f3666aae700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f36600d3710 0x7f36600cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:01.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.703+0000 7f3666aae700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36600d0a80 con 0x7f36600cfa90 2023-06-23T18:37:01.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.703+0000 7f3666aae700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36600cff10 con 0x7f36600d3710 2023-06-23T18:37:01.710 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.703+0000 7f3666aae700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36600b8790 con 0x7f36600b83c0 2023-06-23T18:37:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.704+0000 7f365ffff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f36600d3710 0x7f36600cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.704+0000 7f366504b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f36600cfa90 0x7f366015feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44622/0 (socket says 172.21.15.79:44622) 2023-06-23T18:37:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.704+0000 7f366504b700 1 -- 172.21.15.79:0/1923542965 learned_addr learned my addr 172.21.15.79:0/1923542965 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.704+0000 7f365ffff700 1 -- 172.21.15.79:0/1923542965 >> v2:172.21.15.155:3300/0 conn(0x7f36600b83c0 msgr2=0x7f36600cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.704+0000 7f366484a700 1 --2- 172.21.15.79:0/1923542965 >> v2:172.21.15.155:3300/0 conn(0x7f36600b83c0 0x7f36600cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.704+0000 7f365ffff700 1 --2- 172.21.15.79:0/1923542965 >> v2:172.21.15.155:3300/0 conn(0x7f36600b83c0 0x7f36600cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.704+0000 7f365ffff700 1 -- 172.21.15.79:0/1923542965 >> v1:172.21.15.79:6789/0 conn(0x7f36600cfa90 legacy=0x7f366015feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.704+0000 7f365ffff700 1 -- 172.21.15.79:0/1923542965 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36600b4e70 con 0x7f36600d3710 2023-06-23T18:37:01.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.704+0000 7f365f7fe700 1 -- 172.21.15.79:0/1923542965 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147606132 0 0) 0x7f36600d0a80 con 0x7f36600cfa90 2023-06-23T18:37:01.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.704+0000 7f365ffff700 1 --2- 172.21.15.79:0/1923542965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36600d3710 0x7f36600cf550 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f36600cf520 tx=0x7f364c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a38a53af110e56 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:01.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.704+0000 7f365f7fe700 1 -- 172.21.15.79:0/1923542965 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c011830 con 0x7f36600d3710 2023-06-23T18:37:01.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.704+0000 7f365f7fe700 1 -- 172.21.15.79:0/1923542965 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c011990 con 0x7f36600d3710 2023-06-23T18:37:01.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.704+0000 7f365f7fe700 1 -- 172.21.15.79:0/1923542965 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c011d20 con 0x7f36600d3710 2023-06-23T18:37:01.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.705+0000 7f3666aae700 1 -- 172.21.15.79:0/1923542965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36600d3710 msgr2=0x7f36600cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:01.712 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.705+0000 7f3666aae700 1 --2- 172.21.15.79:0/1923542965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36600d3710 0x7f36600cf550 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f36600cf520 tx=0x7f364c009b20 comp rx=0 tx=0).stop 2023-06-23T18:37:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.705+0000 7f3666aae700 1 -- 172.21.15.79:0/1923542965 shutdown_connections 2023-06-23T18:37:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.705+0000 7f3666aae700 1 --2- 172.21.15.79:0/1923542965 >> v2:172.21.15.155:3300/0 conn(0x7f36600b83c0 0x7f36600cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.705+0000 7f3666aae700 1 --2- 172.21.15.79:0/1923542965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36600d3710 0x7f36600cf550 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.705+0000 7f3666aae700 1 -- 172.21.15.79:0/1923542965 >> 172.21.15.79:0/1923542965 conn(0x7f366006b350 msgr2=0x7f366006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.705+0000 7f3666aae700 1 -- 172.21.15.79:0/1923542965 shutdown_connections 2023-06-23T18:37:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.705+0000 7f3666aae700 1 -- 172.21.15.79:0/1923542965 wait complete. 2023-06-23T18:37:01.715 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.706+0000 7f3666aae700 1 Processor -- start 2023-06-23T18:37:01.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.706+0000 7f3666aae700 1 -- start start 2023-06-23T18:37:01.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f3666aae700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36600b83c0 0x7f3660154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:01.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f3666aae700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36600cfa90 0x7f366015c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:01.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f3666aae700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36600d3710 0x7f366015ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:01.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f3666aae700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36601545a0 con 0x7f36600d3710 2023-06-23T18:37:01.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f3666aae700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3660154a30 con 0x7f36600cfa90 2023-06-23T18:37:01.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f3666aae700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366015d480 con 0x7f36600b83c0 2023-06-23T18:37:01.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f366504b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36600d3710 0x7f366015ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:01.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f366504b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36600d3710 0x7f366015ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41586/0 (socket says 172.21.15.79:41586) 2023-06-23T18:37:01.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f366504b700 1 -- 172.21.15.79:0/3345776431 learned_addr learned my addr 172.21.15.79:0/3345776431 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:01.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f365ffff700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36600cfa90 0x7f366015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:01.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f366484a700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36600b83c0 0x7f3660154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:01.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f366504b700 1 -- 172.21.15.79:0/3345776431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36600b83c0 msgr2=0x7f3660154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:01.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f366504b700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36600b83c0 0x7f3660154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:01.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f366504b700 1 -- 172.21.15.79:0/3345776431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36600cfa90 msgr2=0x7f366015c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:01.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f366504b700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36600cfa90 0x7f366015c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:01.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f366504b700 1 -- 172.21.15.79:0/3345776431 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c0094a0 con 0x7f36600d3710 2023-06-23T18:37:01.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f365ffff700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36600cfa90 0x7f366015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:01.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f366484a700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36600b83c0 0x7f3660154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:01.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.707+0000 7f366504b700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36600d3710 0x7f366015ce70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f365800cba0 tx=0x7f365800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:01.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.708+0000 7f365d7fa700 1 -- 172.21.15.79:0/3345776431 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800ab80 con 0x7f36600d3710 2023-06-23T18:37:01.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.708+0000 7f365d7fa700 1 -- 172.21.15.79:0/3345776431 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365800ad00 con 0x7f36600d3710 2023-06-23T18:37:01.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.708+0000 7f3666aae700 1 -- 172.21.15.79:0/3345776431 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366015d760 con 0x7f36600d3710 2023-06-23T18:37:01.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.708+0000 7f365d7fa700 1 -- 172.21.15.79:0/3345776431 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658007310 con 0x7f36600d3710 2023-06-23T18:37:01.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.708+0000 7f3666aae700 1 -- 172.21.15.79:0/3345776431 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366015dfa0 con 0x7f36600d3710 2023-06-23T18:37:01.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.709+0000 7f3666aae700 1 -- 172.21.15.79:0/3345776431 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366015ac10 con 0x7f36600d3710 2023-06-23T18:37:01.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.709+0000 7f365d7fa700 1 -- 172.21.15.79:0/3345776431 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3658007470 con 0x7f36600d3710 2023-06-23T18:37:01.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.709+0000 7f365d7fa700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f364406ace0 0x7f364406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:01.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.710+0000 7f365d7fa700 1 -- 172.21.15.79:0/3345776431 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f36580892b0 con 0x7f36600d3710 2023-06-23T18:37:01.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.710+0000 7f366484a700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f364406ace0 0x7f364406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:01.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.710+0000 7f366484a700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f364406ace0 0x7f364406d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3654006fd0 tx=0x7f3654008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:01.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.714+0000 7f365d7fa700 1 -- 172.21.15.79:0/3345776431 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f36580543f0 con 0x7f36600d3710 2023-06-23T18:37:01.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.943+0000 7f3666aae700 1 -- 172.21.15.79:0/3345776431 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f366015a930 con 0x7f36600d3710 2023-06-23T18:37:01.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.944+0000 7f365d7fa700 1 -- 172.21.15.79:0/3345776431 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4677 (secure 0 0 0) 0x7f3658057a10 con 0x7f36600d3710 2023-06-23T18:37:01.949 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:01.949 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3101032943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":3101032943},{"type":"v1","addr":"172.21.15.79:6837","nonce":3101032943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2056827644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2056827644},{"type":"v1","addr":"172.21.15.155:6833","nonce":2056827644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1835571378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1835571378},{"type":"v1","addr":"172.21.15.155:6835","nonce":1835571378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:36:58.545517+0000","modified":"2023-06-23T18:37:01.557968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6836/1030786659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1030786659},{"type":"v1","addr":"172.21.15.79:6836","nonce":1030786659}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-23T18:37:01.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.946+0000 7f3666aae700 1 -- 172.21.15.79:0/3345776431 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f364406ace0 msgr2=0x7f364406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:01.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.946+0000 7f3666aae700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f364406ace0 0x7f364406d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3654006fd0 tx=0x7f3654008040 comp rx=0 tx=0).stop 2023-06-23T18:37:01.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.946+0000 7f3666aae700 1 -- 172.21.15.79:0/3345776431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36600d3710 msgr2=0x7f366015ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:01.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.946+0000 7f3666aae700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36600d3710 0x7f366015ce70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f365800cba0 tx=0x7f365800cf60 comp rx=0 tx=0).stop 2023-06-23T18:37:01.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.946+0000 7f3666aae700 1 -- 172.21.15.79:0/3345776431 shutdown_connections 2023-06-23T18:37:01.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.946+0000 7f3666aae700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36600d3710 0x7f366015ce70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:01.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.946+0000 7f3666aae700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36600b83c0 0x7f3660154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:01.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.946+0000 7f3666aae700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f364406ace0 0x7f364406d190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:01.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.946+0000 7f3666aae700 1 --2- 172.21.15.79:0/3345776431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36600cfa90 0x7f366015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:01.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.946+0000 7f3666aae700 1 -- 172.21.15.79:0/3345776431 >> 172.21.15.79:0/3345776431 conn(0x7f366006b350 msgr2=0x7f36600b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:01.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.946+0000 7f3666aae700 1 -- 172.21.15.79:0/3345776431 shutdown_connections 2023-06-23T18:37:01.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:01.946+0000 7f3666aae700 1 -- 172.21.15.79:0/3345776431 wait complete. 2023-06-23T18:37:01.953 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 16 2023-06-23T18:37:01.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:37:02.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.078+0000 7f540666b700 1 Processor -- start 2023-06-23T18:37:02.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.078+0000 7f540666b700 1 -- start start 2023-06-23T18:37:02.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f540666b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5400159170 0x7f54000cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f540666b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54000ce7f0 0x7f54000cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f540666b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540015cd50 con 0x7f54000cd900 2023-06-23T18:37:02.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f540666b700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54000cdd80 con 0x7f5400159170 2023-06-23T18:37:02.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f540666b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540015cfa0 con 0x7f54000ce7f0 2023-06-23T18:37:02.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f5404c08700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54000ce7f0 0x7f54000cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f53fffff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5400159170 0x7f54000cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f53ff7fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f54000cd900 0x7f54000ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:52900/0 (socket says 172.21.15.79:52900) 2023-06-23T18:37:02.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f5404c08700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54000ce7f0 0x7f54000cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:34364/0 (socket says 172.21.15.79:34364) 2023-06-23T18:37:02.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f5404c08700 1 -- 172.21.15.79:0/1955082063 learned_addr learned my addr 172.21.15.79:0/1955082063 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:02.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f53fffff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5400159170 0x7f54000cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:47816/0 (socket says 172.21.15.79:47816) 2023-06-23T18:37:02.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f5404c08700 1 -- 172.21.15.79:0/1955082063 >> v2:172.21.15.155:3300/0 conn(0x7f5400159170 msgr2=0x7f54000cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:02.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f5404c08700 1 --2- 172.21.15.79:0/1955082063 >> v2:172.21.15.155:3300/0 conn(0x7f5400159170 0x7f54000cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f5404c08700 1 -- 172.21.15.79:0/1955082063 >> v1:172.21.15.155:6789/0 conn(0x7f54000cd900 legacy=0x7f54000ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f5404c08700 1 -- 172.21.15.79:0/1955082063 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540015d280 con 0x7f54000ce7f0 2023-06-23T18:37:02.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.079+0000 7f53fffff700 1 --2- 172.21.15.79:0/1955082063 >> v2:172.21.15.155:3300/0 conn(0x7f5400159170 0x7f54000cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:02.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.080+0000 7f5404c08700 1 --2- 172.21.15.79:0/1955082063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54000ce7f0 0x7f54000cc1d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009e50 tx=0x7f53f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79fd17274603ad4b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:02.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.080+0000 7f53feffd700 1 -- 172.21.15.79:0/1955082063 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f800e070 con 0x7f54000ce7f0 2023-06-23T18:37:02.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.080+0000 7f53feffd700 1 -- 172.21.15.79:0/1955082063 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f800b910 con 0x7f54000ce7f0 2023-06-23T18:37:02.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.080+0000 7f53feffd700 1 -- 172.21.15.79:0/1955082063 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f800bcc0 con 0x7f54000ce7f0 2023-06-23T18:37:02.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.080+0000 7f540666b700 1 -- 172.21.15.79:0/1955082063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54000ce7f0 msgr2=0x7f54000cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.080+0000 7f540666b700 1 --2- 172.21.15.79:0/1955082063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54000ce7f0 0x7f54000cc1d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009e50 tx=0x7f53f800b330 comp rx=0 tx=0).stop 2023-06-23T18:37:02.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.080+0000 7f540666b700 1 -- 172.21.15.79:0/1955082063 shutdown_connections 2023-06-23T18:37:02.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.080+0000 7f540666b700 1 --2- 172.21.15.79:0/1955082063 >> v2:172.21.15.155:3300/0 conn(0x7f5400159170 0x7f54000cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.080+0000 7f540666b700 1 --2- 172.21.15.79:0/1955082063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54000ce7f0 0x7f54000cc1d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.080+0000 7f540666b700 1 -- 172.21.15.79:0/1955082063 >> 172.21.15.79:0/1955082063 conn(0x7f540006d660 msgr2=0x7f540006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:02.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.080+0000 7f540666b700 1 -- 172.21.15.79:0/1955082063 shutdown_connections 2023-06-23T18:37:02.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.080+0000 7f540666b700 1 -- 172.21.15.79:0/1955082063 wait complete. 2023-06-23T18:37:02.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.082+0000 7f540666b700 1 Processor -- start 2023-06-23T18:37:02.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.082+0000 7f540666b700 1 -- start start 2023-06-23T18:37:02.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.082+0000 7f540666b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54000cd900 0x7f54000b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.082+0000 7f540666b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54000ce7f0 0x7f54000c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.082+0000 7f540666b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5400159170 0x7f54000b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.082+0000 7f540666b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5400159830 con 0x7f54000cd900 2023-06-23T18:37:02.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.082+0000 7f540666b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5400159aa0 con 0x7f54000ce7f0 2023-06-23T18:37:02.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.082+0000 7f540666b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54000b4dd0 con 0x7f5400159170 2023-06-23T18:37:02.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f53fffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54000cd900 0x7f54000b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f53fffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54000cd900 0x7f54000b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41596/0 (socket says 172.21.15.79:41596) 2023-06-23T18:37:02.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f53fffff700 1 -- 172.21.15.79:0/343282269 learned_addr learned my addr 172.21.15.79:0/343282269 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:02.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f5404c08700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5400159170 0x7f54000b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f53ff7fe700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54000ce7f0 0x7f54000c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f53fffff700 1 -- 172.21.15.79:0/343282269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5400159170 msgr2=0x7f54000b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f53fffff700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5400159170 0x7f54000b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f53fffff700 1 -- 172.21.15.79:0/343282269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54000ce7f0 msgr2=0x7f54000c27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f53fffff700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54000ce7f0 0x7f54000c27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f53fffff700 1 -- 172.21.15.79:0/343282269 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f00094a0 con 0x7f54000cd900 2023-06-23T18:37:02.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f5404c08700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5400159170 0x7f54000b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:02.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f53fffff700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54000cd900 0x7f54000b47f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f53f000ec10 tx=0x7f53f0009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:02.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f53fcff9700 1 -- 172.21.15.79:0/343282269 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f000c940 con 0x7f54000cd900 2023-06-23T18:37:02.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f53fcff9700 1 -- 172.21.15.79:0/343282269 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f000caa0 con 0x7f54000cd900 2023-06-23T18:37:02.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f540666b700 1 -- 172.21.15.79:0/343282269 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f8009b30 con 0x7f54000cd900 2023-06-23T18:37:02.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f53fcff9700 1 -- 172.21.15.79:0/343282269 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f000ce10 con 0x7f54000cd900 2023-06-23T18:37:02.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.083+0000 7f540666b700 1 -- 172.21.15.79:0/343282269 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54000b5390 con 0x7f54000cd900 2023-06-23T18:37:02.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.084+0000 7f540666b700 1 -- 172.21.15.79:0/343282269 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54000732a0 con 0x7f54000cd900 2023-06-23T18:37:02.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.085+0000 7f53fcff9700 1 -- 172.21.15.79:0/343282269 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f53f0013470 con 0x7f54000cd900 2023-06-23T18:37:02.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.085+0000 7f53fcff9700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f53ec06ace0 0x7f53ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.085+0000 7f53fcff9700 1 -- 172.21.15.79:0/343282269 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f53f0089a70 con 0x7f54000cd900 2023-06-23T18:37:02.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.085+0000 7f53ff7fe700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f53ec06ace0 0x7f53ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.085+0000 7f53ff7fe700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f53ec06ace0 0x7f53ec06d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f54000cbc30 tx=0x7f53f4007d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:02.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.089+0000 7f53fcff9700 1 -- 172.21.15.79:0/343282269 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f53f0054a90 con 0x7f54000cd900 2023-06-23T18:37:02.132 INFO:tasks.ceph.mds.c.smithi079.stderr:2023-06-23T18:37:02.128+0000 7f7fbcff8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-23T18:37:02.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.281+0000 7f540666b700 1 -- 172.21.15.79:0/343282269 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5400159d80 con 0x7f54000cd900 2023-06-23T18:37:02.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.283+0000 7f53fcff9700 1 -- 172.21.15.79:0/343282269 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+18305 (secure 0 0 0) 0x7f53f0021090 con 0x7f54000cd900 2023-06-23T18:37:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:02.288 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":34,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:36:59.547297+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:36:56.964534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-23T18:36:57.507393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:37:02.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.285+0000 7f540666b700 1 -- 172.21.15.79:0/343282269 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f53ec06ace0 msgr2=0x7f53ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.285+0000 7f540666b700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f53ec06ace0 0x7f53ec06d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f54000cbc30 tx=0x7f53f4007d10 comp rx=0 tx=0).stop 2023-06-23T18:37:02.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.285+0000 7f540666b700 1 -- 172.21.15.79:0/343282269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54000cd900 msgr2=0x7f54000b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.285+0000 7f540666b700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54000cd900 0x7f54000b47f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f53f000ec10 tx=0x7f53f0009ec0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.285+0000 7f540666b700 1 -- 172.21.15.79:0/343282269 shutdown_connections 2023-06-23T18:37:02.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.285+0000 7f540666b700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5400159170 0x7f54000b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.285+0000 7f540666b700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54000cd900 0x7f54000b47f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.285+0000 7f540666b700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f53ec06ace0 0x7f53ec06d190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.285+0000 7f540666b700 1 --2- 172.21.15.79:0/343282269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54000ce7f0 0x7f54000c27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.285+0000 7f540666b700 1 -- 172.21.15.79:0/343282269 >> 172.21.15.79:0/343282269 conn(0x7f540006d660 msgr2=0x7f54000ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:02.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.285+0000 7f540666b700 1 -- 172.21.15.79:0/343282269 shutdown_connections 2023-06-23T18:37:02.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.285+0000 7f540666b700 1 -- 172.21.15.79:0/343282269 wait complete. 2023-06-23T18:37:02.302 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-06-23T18:37:02.302 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:37:02.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.417+0000 7f99211d8700 1 Processor -- start 2023-06-23T18:37:02.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.417+0000 7f99211d8700 1 -- start start 2023-06-23T18:37:02.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.418+0000 7f99211d8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f991c0b1270 0x7f991c15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.418+0000 7f99211d8700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f991c0aee30 0x7f991c15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.418+0000 7f99211d8700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c1608f0 con 0x7f991c071490 2023-06-23T18:37:02.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.418+0000 7f99211d8700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c160e20 con 0x7f991c0b1270 2023-06-23T18:37:02.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.418+0000 7f99211d8700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c0cd4f0 con 0x7f991c0aee30 2023-06-23T18:37:02.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.418+0000 7f991a59c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f991c0b1270 0x7f991c15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.418+0000 7f991ad9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f991c071490 0x7f991c071860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44646/0 (socket says 172.21.15.79:44646) 2023-06-23T18:37:02.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.418+0000 7f991ad9d700 1 -- 172.21.15.79:0/3817516688 learned_addr learned my addr 172.21.15.79:0/3817516688 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:02.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.418+0000 7f991b59e700 1 --2- 172.21.15.79:0/3817516688 >> v2:172.21.15.155:3300/0 conn(0x7f991c0aee30 0x7f991c15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.418+0000 7f9919d9b700 1 -- 172.21.15.79:0/3817516688 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55168516 0 0) 0x7f991c1608f0 con 0x7f991c071490 2023-06-23T18:37:02.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.418+0000 7f9919d9b700 1 -- 172.21.15.79:0/3817516688 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99040035b0 con 0x7f991c071490 2023-06-23T18:37:02.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.418+0000 7f991a59c700 1 -- 172.21.15.79:0/3817516688 >> v2:172.21.15.155:3300/0 conn(0x7f991c0aee30 msgr2=0x7f991c15cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:02.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.418+0000 7f991a59c700 1 --2- 172.21.15.79:0/3817516688 >> v2:172.21.15.155:3300/0 conn(0x7f991c0aee30 0x7f991c15cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.418+0000 7f991a59c700 1 -- 172.21.15.79:0/3817516688 >> v1:172.21.15.79:6789/0 conn(0x7f991c071490 legacy=0x7f991c071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.419+0000 7f991a59c700 1 -- 172.21.15.79:0/3817516688 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c160b40 con 0x7f991c0b1270 2023-06-23T18:37:02.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.419+0000 7f991a59c700 1 --2- 172.21.15.79:0/3817516688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f991c0b1270 0x7f991c15c9f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f991c071290 tx=0x7f990c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e9fc90e5e3fec1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:02.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.419+0000 7f9919d9b700 1 -- 172.21.15.79:0/3817516688 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c011830 con 0x7f991c0b1270 2023-06-23T18:37:02.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.419+0000 7f9919d9b700 1 -- 172.21.15.79:0/3817516688 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c011990 con 0x7f991c0b1270 2023-06-23T18:37:02.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.419+0000 7f9919d9b700 1 -- 172.21.15.79:0/3817516688 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c011d20 con 0x7f991c0b1270 2023-06-23T18:37:02.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.419+0000 7f99211d8700 1 -- 172.21.15.79:0/3817516688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f991c0b1270 msgr2=0x7f991c15c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.419+0000 7f99211d8700 1 --2- 172.21.15.79:0/3817516688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f991c0b1270 0x7f991c15c9f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f991c071290 tx=0x7f990c009b20 comp rx=0 tx=0).stop 2023-06-23T18:37:02.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.419+0000 7f99211d8700 1 -- 172.21.15.79:0/3817516688 shutdown_connections 2023-06-23T18:37:02.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.419+0000 7f99211d8700 1 --2- 172.21.15.79:0/3817516688 >> v2:172.21.15.155:3300/0 conn(0x7f991c0aee30 0x7f991c15cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.419+0000 7f99211d8700 1 --2- 172.21.15.79:0/3817516688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f991c0b1270 0x7f991c15c9f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.419+0000 7f99211d8700 1 -- 172.21.15.79:0/3817516688 >> 172.21.15.79:0/3817516688 conn(0x7f991c06d660 msgr2=0x7f991c06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:02.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.419+0000 7f99211d8700 1 -- 172.21.15.79:0/3817516688 shutdown_connections 2023-06-23T18:37:02.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.419+0000 7f99211d8700 1 -- 172.21.15.79:0/3817516688 wait complete. 2023-06-23T18:37:02.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.421+0000 7f99211d8700 1 Processor -- start 2023-06-23T18:37:02.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.421+0000 7f99211d8700 1 -- start start 2023-06-23T18:37:02.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f99211d8700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f991c071490 0x7f991c148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f99211d8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f991c0aee30 0x7f991c148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f99211d8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f991c0b1270 0x7f991c0c3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f99211d8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0c6190 con 0x7f991c0aee30 2023-06-23T18:37:02.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f99211d8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0c9f50 con 0x7f991c0b1270 2023-06-23T18:37:02.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f99211d8700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0ca230 con 0x7f991c071490 2023-06-23T18:37:02.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f991a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f991c0aee30 0x7f991c148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f991a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f991c0aee30 0x7f991c148160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41612/0 (socket says 172.21.15.79:41612) 2023-06-23T18:37:02.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f991a59c700 1 -- 172.21.15.79:0/3767416836 learned_addr learned my addr 172.21.15.79:0/3767416836 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:02.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f991b59e700 1 --2- 172.21.15.79:0/3767416836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f991c0b1270 0x7f991c0c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f991ad9d700 1 --2- 172.21.15.79:0/3767416836 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f991c071490 0x7f991c148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f991a59c700 1 -- 172.21.15.79:0/3767416836 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f991c071490 msgr2=0x7f991c148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f991a59c700 1 --2- 172.21.15.79:0/3767416836 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f991c071490 0x7f991c148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f991a59c700 1 -- 172.21.15.79:0/3767416836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f991c0b1270 msgr2=0x7f991c0c3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f991a59c700 1 --2- 172.21.15.79:0/3767416836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f991c0b1270 0x7f991c0c3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f991a59c700 1 -- 172.21.15.79:0/3767416836 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c0094a0 con 0x7f991c0aee30 2023-06-23T18:37:02.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f991a59c700 1 --2- 172.21.15.79:0/3767416836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f991c0aee30 0x7f991c148160 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f990c009b60 tx=0x7f990c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:02.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f99037fe700 1 -- 172.21.15.79:0/3767416836 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c004550 con 0x7f991c0aee30 2023-06-23T18:37:02.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f99037fe700 1 -- 172.21.15.79:0/3767416836 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c01e070 con 0x7f991c0aee30 2023-06-23T18:37:02.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f99211d8700 1 -- 172.21.15.79:0/3767416836 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c0ca510 con 0x7f991c0aee30 2023-06-23T18:37:02.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f99037fe700 1 -- 172.21.15.79:0/3767416836 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0038e0 con 0x7f991c0aee30 2023-06-23T18:37:02.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.422+0000 7f99211d8700 1 -- 172.21.15.79:0/3767416836 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c1486a0 con 0x7f991c0aee30 2023-06-23T18:37:02.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.423+0000 7f99211d8700 1 -- 172.21.15.79:0/3767416836 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c0caa30 con 0x7f991c0aee30 2023-06-23T18:37:02.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.424+0000 7f99037fe700 1 -- 172.21.15.79:0/3767416836 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f990c003a40 con 0x7f991c0aee30 2023-06-23T18:37:02.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.424+0000 7f99037fe700 1 --2- 172.21.15.79:0/3767416836 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f990806ace0 0x7f990806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.424+0000 7f99037fe700 1 -- 172.21.15.79:0/3767416836 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f990c08a280 con 0x7f991c0aee30 2023-06-23T18:37:02.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.424+0000 7f991ad9d700 1 --2- 172.21.15.79:0/3767416836 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f990806ace0 0x7f990806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.424+0000 7f991ad9d700 1 --2- 172.21.15.79:0/3767416836 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f990806ace0 0x7f990806d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9910007fd0 tx=0x7f9910009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:02.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.428+0000 7f99037fe700 1 -- 172.21.15.79:0/3767416836 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f991c0caa30 con 0x7f991c0aee30 2023-06-23T18:37:02.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.612+0000 7f99211d8700 1 -- 172.21.15.79:0/3767416836 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f991c0ca0c0 con 0x7f990806ace0 2023-06-23T18:37:02.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.620+0000 7f99037fe700 1 -- 172.21.15.79:0/3767416836 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296540 (secure 0 0 0) 0x7f991c0ca0c0 con 0x7f990806ace0 2023-06-23T18:37:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:02.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.624+0000 7f99211d8700 1 -- 172.21.15.79:0/3767416836 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f990806ace0 msgr2=0x7f990806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.624+0000 7f99211d8700 1 --2- 172.21.15.79:0/3767416836 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f990806ace0 0x7f990806d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9910007fd0 tx=0x7f9910009040 comp rx=0 tx=0).stop 2023-06-23T18:37:02.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.624+0000 7f99211d8700 1 -- 172.21.15.79:0/3767416836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f991c0aee30 msgr2=0x7f991c148160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.624+0000 7f99211d8700 1 --2- 172.21.15.79:0/3767416836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f991c0aee30 0x7f991c148160 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f990c009b60 tx=0x7f990c004750 comp rx=0 tx=0).stop 2023-06-23T18:37:02.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.624+0000 7f99211d8700 1 -- 172.21.15.79:0/3767416836 shutdown_connections 2023-06-23T18:37:02.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.624+0000 7f99211d8700 1 --2- 172.21.15.79:0/3767416836 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f990806ace0 0x7f990806d190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.624+0000 7f99211d8700 1 --2- 172.21.15.79:0/3767416836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f991c0b1270 0x7f991c0c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.624+0000 7f99211d8700 1 --2- 172.21.15.79:0/3767416836 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f991c071490 0x7f991c148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.624+0000 7f99211d8700 1 --2- 172.21.15.79:0/3767416836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f991c0aee30 0x7f991c148160 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.624+0000 7f99211d8700 1 -- 172.21.15.79:0/3767416836 >> 172.21.15.79:0/3767416836 conn(0x7f991c06d660 msgr2=0x7f991c15b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.624+0000 7f99211d8700 1 -- 172.21.15.79:0/3767416836 shutdown_connections 2023-06-23T18:37:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.624+0000 7f99211d8700 1 -- 172.21.15.79:0/3767416836 wait complete. 2023-06-23T18:37:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-23T18:37:02.641 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-06-23T18:37:00.708909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219388,"kb_used_data":3772,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755332,"statfs":{"total":773094113280,"available":772869459968,"internally_reserved":0,"allocated":3862528,"data_stored":2245757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000585"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-06-23T18:36:52.329026+0000","last_change":"2023-06-23T18:35:48.920009+0000","last_active":"2023-06-23T18:36:52.329026+0000","last_peered":"2023-06-23T18:36:52.329026+0000","last_clean":"2023-06-23T18:36:52.329026+0000","last_became_active":"2023-06-23T18:35:48.919686+0000","last_became_peered":"2023-06-23T18:35:48.919686+0000","last_unstale":"2023-06-23T18:36:52.329026+0000","last_undegraded":"2023-06-23T18:36:52.329026+0000","last_fullsized":"2023-06-23T18:36:52.329026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:59:14.940430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-06-23T18:36:52.379140+0000","last_change":"2023-06-23T18:35:47.962164+0000","last_active":"2023-06-23T18:36:52.379140+0000","last_peered":"2023-06-23T18:36:52.379140+0000","last_clean":"2023-06-23T18:36:52.379140+0000","last_became_active":"2023-06-23T18:35:47.961790+0000","last_became_peered":"2023-06-23T18:35:47.961790+0000","last_unstale":"2023-06-23T18:36:52.379140+0000","last_undegraded":"2023-06-23T18:36:52.379140+0000","last_fullsized":"2023-06-23T18:36:52.379140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:11:36.309515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-06-23T18:36:52.379040+0000","last_change":"2023-06-23T18:35:47.965867+0000","last_active":"2023-06-23T18:36:52.379040+0000","last_peered":"2023-06-23T18:36:52.379040+0000","last_clean":"2023-06-23T18:36:52.379040+0000","last_became_active":"2023-06-23T18:35:47.965580+0000","last_became_peered":"2023-06-23T18:35:47.965580+0000","last_unstale":"2023-06-23T18:36:52.379040+0000","last_undegraded":"2023-06-23T18:36:52.379040+0000","last_fullsized":"2023-06-23T18:36:52.379040+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:50:38.068740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":185,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-06-23T18:36:52.379266+0000","last_change":"2023-06-23T18:35:47.968534+0000","last_active":"2023-06-23T18:36:52.379266+0000","last_peered":"2023-06-23T18:36:52.379266+0000","last_clean":"2023-06-23T18:36:52.379266+0000","last_became_active":"2023-06-23T18:35:47.967508+0000","last_became_peered":"2023-06-23T18:35:47.967508+0000","last_unstale":"2023-06-23T18:36:52.379266+0000","last_undegraded":"2023-06-23T18:36:52.379266+0000","last_fullsized":"2023-06-23T18:36:52.379266+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:08:13.778628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-06-23T18:36:52.379287+0000","last_change":"2023-06-23T18:35:48.736407+0000","last_active":"2023-06-23T18:36:52.379287+0000","last_peered":"2023-06-23T18:36:52.379287+0000","last_clean":"2023-06-23T18:36:52.379287+0000","last_became_active":"2023-06-23T18:35:48.735621+0000","last_became_peered":"2023-06-23T18:35:48.735621+0000","last_unstale":"2023-06-23T18:36:52.379287+0000","last_undegraded":"2023-06-23T18:36:52.379287+0000","last_fullsized":"2023-06-23T18:36:52.379287+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:33:20.925859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-06-23T18:36:52.379252+0000","last_change":"2023-06-23T18:35:48.736449+0000","last_active":"2023-06-23T18:36:52.379252+0000","last_peered":"2023-06-23T18:36:52.379252+0000","last_clean":"2023-06-23T18:36:52.379252+0000","last_became_active":"2023-06-23T18:35:48.735664+0000","last_became_peered":"2023-06-23T18:35:48.735664+0000","last_unstale":"2023-06-23T18:36:52.379252+0000","last_undegraded":"2023-06-23T18:36:52.379252+0000","last_fullsized":"2023-06-23T18:36:52.379252+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:07:02.472442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-06-23T18:36:52.329092+0000","last_change":"2023-06-23T18:35:48.735713+0000","last_active":"2023-06-23T18:36:52.329092+0000","last_peered":"2023-06-23T18:36:52.329092+0000","last_clean":"2023-06-23T18:36:52.329092+0000","last_became_active":"2023-06-23T18:35:48.735236+0000","last_became_peered":"2023-06-23T18:35:48.735236+0000","last_unstale":"2023-06-23T18:36:52.329092+0000","last_undegraded":"2023-06-23T18:36:52.329092+0000","last_fullsized":"2023-06-23T18:36:52.329092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:36:08.773602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-06-23T18:36:52.366169+0000","last_change":"2023-06-23T18:35:48.735631+0000","last_active":"2023-06-23T18:36:52.366169+0000","last_peered":"2023-06-23T18:36:52.366169+0000","last_clean":"2023-06-23T18:36:52.366169+0000","last_became_active":"2023-06-23T18:35:48.735255+0000","last_became_peered":"2023-06-23T18:35:48.735255+0000","last_unstale":"2023-06-23T18:36:52.366169+0000","last_undegraded":"2023-06-23T18:36:52.366169+0000","last_fullsized":"2023-06-23T18:36:52.366169+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:53:52.233934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:58.143797+0000","last_change":"2023-06-23T18:36:58.143797+0000","last_active":"2023-06-23T18:36:58.143797+0000","last_peered":"2023-06-23T18:36:58.143797+0000","last_clean":"2023-06-23T18:36:58.143797+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:58.143797+0000","last_undegraded":"2023-06-23T18:36:58.143797+0000","last_fullsized":"2023-06-23T18:36:58.143797+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-23T18:36:57.137463+0000","last_change":"2023-06-23T18:36:57.137463+0000","last_active":"2023-06-23T18:36:57.137463+0000","last_peered":"2023-06-23T18:36:57.137463+0000","last_clean":"2023-06-23T18:36:57.137463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-23T18:36:57.137463+0000","last_undegraded":"2023-06-23T18:36:57.137463+0000","last_fullsized":"2023-06-23T18:36:57.137463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-06-23T18:36:52.366169+0000","last_change":"2023-06-23T18:35:47.970359+0000","last_active":"2023-06-23T18:36:52.366169+0000","last_peered":"2023-06-23T18:36:52.366169+0000","last_clean":"2023-06-23T18:36:52.366169+0000","last_became_active":"2023-06-23T18:35:47.970118+0000","last_became_peered":"2023-06-23T18:35:47.970118+0000","last_unstale":"2023-06-23T18:36:52.366169+0000","last_undegraded":"2023-06-23T18:36:52.366169+0000","last_fullsized":"2023-06-23T18:36:52.366169+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:42:31.807793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":925696,"data_stored":722645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":11.071,"5min":11.071,"15min":11.071},"last":0.39800000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":11.118,"5min":11.118,"15min":11.118},"last":0.55300000000000005},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":11.103999999999999,"5min":11.103999999999999,"15min":11.103999999999999},"last":0.53400000000000003}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.371},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":0.501}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.442}]},{"osd":5,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.59699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":335872,"data_stored":133638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.753},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.67500000000000004}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54900000000000004}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.501}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.68700000000000006}]},{"osd":7,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.443},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.52100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":929792,"data_stored":724006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.27800000000000002},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.8680000000000003,"5min":7.8680000000000003,"15min":7.8680000000000003},"last":0.34000000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.6429999999999998,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":9.4209999999999994,"5min":9.4209999999999994,"15min":9.4209999999999994},"last":0.317}]},{"osd":3,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":5.6790000000000003},"last":0.35599999999999998},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":7.9039999999999999,"5min":7.9039999999999999,"15min":7.9039999999999999},"last":0.30499999999999999}]},{"osd":4,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":11.385999999999999,"5min":11.385999999999999,"15min":11.385999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.63900000000000001}]},{"osd":5,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":7.9649999999999999,"5min":7.9649999999999999,"15min":7.9649999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":11.259,"5min":11.259,"15min":11.259},"last":0.64800000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":7.9909999999999997,"5min":7.9909999999999997,"15min":7.9909999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":11.346,"5min":11.346,"15min":11.346},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":7.9359999999999999,"5min":7.9359999999999999,"15min":7.9359999999999999},"last":0.52300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":324,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":331776,"data_stored":132277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":16.004999999999999,"5min":16.004999999999999,"15min":16.004999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":16.073,"5min":16.073,"15min":16.073},"last":0.30199999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":8.2460000000000004,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":8.2129999999999992,"5min":8.2129999999999992,"15min":8.2129999999999992},"last":0.34200000000000003}]},{"osd":3,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.254},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":16.036999999999999,"5min":16.036999999999999,"15min":16.036999999999999},"last":0.33100000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":8.0600000000000005,"5min":8.0600000000000005,"15min":8.0600000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":8.2739999999999991,"5min":8.2739999999999991,"15min":8.2739999999999991},"last":0.48399999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":8.3320000000000007,"5min":8.3320000000000007,"15min":8.3320000000000007},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":8.0299999999999994,"5min":8.0299999999999994,"15min":8.0299999999999994},"last":0.53600000000000003}]},{"osd":6,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.3030000000000008,"5min":8.3030000000000008,"15min":8.3030000000000008},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":8.3800000000000008,"5min":8.3800000000000008,"15min":8.3800000000000008},"last":0.495}]},{"osd":7,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":8.3580000000000005,"5min":8.3580000000000005,"15min":8.3580000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":8.0839999999999996,"5min":8.0839999999999996,"15min":8.0839999999999996},"last":0.47399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":324,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":331776,"data_stored":132277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.314}]},{"osd":3,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":5.0899999999999999,"5min":5.0899999999999999,"15min":5.0899999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.33700000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.436}]},{"osd":5,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.42499999999999999}]},{"osd":6,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.44700000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.498}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":335872,"data_stored":133638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":4.359,"5min":4.359,"15min":4.359},"last":0.27600000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.249}]},{"osd":2,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.31}]},{"osd":4,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68999999999999995}]},{"osd":5,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.58199999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":335872,"data_stored":133638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.54700000000000004}]},{"osd":2,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.70399999999999996}]},{"osd":3,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.77600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":335872,"data_stored":133638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.56699999999999995}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.503},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.42499999999999999}]},{"osd":6,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.623},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.55500000000000005}]},{"osd":7,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:37:02.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:37:02.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.794+0000 7f507fbb6700 1 Processor -- start 2023-06-23T18:37:02.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.795+0000 7f507fbb6700 1 -- start start 2023-06-23T18:37:02.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.795+0000 7f507fbb6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50780b1270 0x7f507815c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.795+0000 7f507fbb6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f50780aee30 0x7f507815cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.795+0000 7f507fbb6700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50781608f0 con 0x7f5078071490 2023-06-23T18:37:02.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.795+0000 7f507fbb6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5078160e20 con 0x7f50780aee30 2023-06-23T18:37:02.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.795+0000 7f507fbb6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50780cd4f0 con 0x7f50780b1270 2023-06-23T18:37:02.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.795+0000 7f507e153700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f50780aee30 0x7f507815cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.796+0000 7f507d952700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5078071490 0x7f5078071860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44660/0 (socket says 172.21.15.79:44660) 2023-06-23T18:37:02.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.796+0000 7f507e153700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f50780aee30 0x7f507815cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41622/0 (socket says 172.21.15.79:41622) 2023-06-23T18:37:02.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.796+0000 7f507d952700 1 -- 172.21.15.79:0/4006270270 learned_addr learned my addr 172.21.15.79:0/4006270270 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:02.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.796+0000 7f507d151700 1 --2- 172.21.15.79:0/4006270270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50780b1270 0x7f507815c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.796+0000 7f507e153700 1 -- 172.21.15.79:0/4006270270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50780b1270 msgr2=0x7f507815c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:02.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.796+0000 7f507e153700 1 --2- 172.21.15.79:0/4006270270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50780b1270 0x7f507815c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.796+0000 7f507e153700 1 -- 172.21.15.79:0/4006270270 >> v1:172.21.15.79:6789/0 conn(0x7f5078071490 legacy=0x7f5078071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.796+0000 7f507c950700 1 -- 172.21.15.79:0/4006270270 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054525972 0 0) 0x7f50781608f0 con 0x7f5078071490 2023-06-23T18:37:02.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.796+0000 7f507e153700 1 -- 172.21.15.79:0/4006270270 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5078160b40 con 0x7f50780aee30 2023-06-23T18:37:02.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.796+0000 7f507e153700 1 --2- 172.21.15.79:0/4006270270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50780aee30 0x7f507815cf30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5070009d90 tx=0x7f507000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbb2bdf758f6269 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:02.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.796+0000 7f507c950700 1 -- 172.21.15.79:0/4006270270 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507000e070 con 0x7f50780aee30 2023-06-23T18:37:02.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.796+0000 7f507c950700 1 -- 172.21.15.79:0/4006270270 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507000bbd0 con 0x7f50780aee30 2023-06-23T18:37:02.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.796+0000 7f507c950700 1 -- 172.21.15.79:0/4006270270 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070012430 con 0x7f50780aee30 2023-06-23T18:37:02.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.797+0000 7f507fbb6700 1 -- 172.21.15.79:0/4006270270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50780aee30 msgr2=0x7f507815cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.797+0000 7f507fbb6700 1 --2- 172.21.15.79:0/4006270270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50780aee30 0x7f507815cf30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5070009d90 tx=0x7f507000b330 comp rx=0 tx=0).stop 2023-06-23T18:37:02.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.797+0000 7f507fbb6700 1 -- 172.21.15.79:0/4006270270 shutdown_connections 2023-06-23T18:37:02.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.797+0000 7f507fbb6700 1 --2- 172.21.15.79:0/4006270270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50780aee30 0x7f507815cf30 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.797+0000 7f507fbb6700 1 --2- 172.21.15.79:0/4006270270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50780b1270 0x7f507815c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.797+0000 7f507fbb6700 1 -- 172.21.15.79:0/4006270270 >> 172.21.15.79:0/4006270270 conn(0x7f507806d660 msgr2=0x7f507806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:02.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.797+0000 7f507fbb6700 1 -- 172.21.15.79:0/4006270270 shutdown_connections 2023-06-23T18:37:02.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.797+0000 7f507fbb6700 1 -- 172.21.15.79:0/4006270270 wait complete. 2023-06-23T18:37:02.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.798+0000 7f507fbb6700 1 Processor -- start 2023-06-23T18:37:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.798+0000 7f507fbb6700 1 -- start start 2023-06-23T18:37:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.799+0000 7f507fbb6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5078071490 0x7f50781536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.799+0000 7f507fbb6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50780aee30 0x7f5078151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.799+0000 7f507fbb6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50780b1270 0x7f5078152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.799+0000 7f507fbb6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50780c7120 con 0x7f50780b1270 2023-06-23T18:37:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.799+0000 7f507fbb6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507814e240 con 0x7f50780aee30 2023-06-23T18:37:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.799+0000 7f507fbb6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5078153c10 con 0x7f5078071490 2023-06-23T18:37:02.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.799+0000 7f507e153700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50780b1270 0x7f5078152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.799+0000 7f507e153700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50780b1270 0x7f5078152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41628/0 (socket says 172.21.15.79:41628) 2023-06-23T18:37:02.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.799+0000 7f507e153700 1 -- 172.21.15.79:0/2450947723 learned_addr learned my addr 172.21.15.79:0/2450947723 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:02.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f507d151700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50780aee30 0x7f5078151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f507d952700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5078071490 0x7f50781536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f507e153700 1 -- 172.21.15.79:0/2450947723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5078071490 msgr2=0x7f50781536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f507e153700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5078071490 0x7f50781536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f507e153700 1 -- 172.21.15.79:0/2450947723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50780aee30 msgr2=0x7f5078151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f507e153700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50780aee30 0x7f5078151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f507e153700 1 -- 172.21.15.79:0/2450947723 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5070009a70 con 0x7f50780b1270 2023-06-23T18:37:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f507d151700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50780aee30 0x7f5078151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f507d952700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5078071490 0x7f50781536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f507e153700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50780b1270 0x7f5078152310 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5078070bc0 tx=0x7f5070004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f506e7fc700 1 -- 172.21.15.79:0/2450947723 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070012430 con 0x7f50780b1270 2023-06-23T18:37:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f506e7fc700 1 -- 172.21.15.79:0/2450947723 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5070020070 con 0x7f50780b1270 2023-06-23T18:37:02.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f507fbb6700 1 -- 172.21.15.79:0/2450947723 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50780c3b00 con 0x7f50780b1270 2023-06-23T18:37:02.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f506e7fc700 1 -- 172.21.15.79:0/2450947723 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070004280 con 0x7f50780b1270 2023-06-23T18:37:02.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.800+0000 7f507fbb6700 1 -- 172.21.15.79:0/2450947723 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50780c6a60 con 0x7f50780b1270 2023-06-23T18:37:02.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.801+0000 7f507fbb6700 1 -- 172.21.15.79:0/2450947723 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50780c68c0 con 0x7f50780b1270 2023-06-23T18:37:02.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.802+0000 7f506e7fc700 1 -- 172.21.15.79:0/2450947723 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f50700043e0 con 0x7f50780b1270 2023-06-23T18:37:02.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.802+0000 7f506e7fc700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f505c06ace0 0x7f505c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:02.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.802+0000 7f506e7fc700 1 -- 172.21.15.79:0/2450947723 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5070022070 con 0x7f50780b1270 2023-06-23T18:37:02.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.802+0000 7f507d952700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f505c06ace0 0x7f505c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:02.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.802+0000 7f507d952700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f505c06ace0 0x7f505c06d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5068000f60 tx=0x7f5068007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:02.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.806+0000 7f506e7fc700 1 -- 172.21.15.79:0/2450947723 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f50780c68c0 con 0x7f50780b1270 2023-06-23T18:37:02.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.989+0000 7f507fbb6700 1 -- 172.21.15.79:0/2450947723 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50780c68c0 con 0x7f505c06ace0 2023-06-23T18:37:03.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:02.997+0000 7f506e7fc700 1 -- 172.21.15.79:0/2450947723 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311349 (secure 0 0 0) 0x7f50780c68c0 con 0x7f505c06ace0 2023-06-23T18:37:03.005 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:03.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.000+0000 7f507fbb6700 1 -- 172.21.15.79:0/2450947723 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f505c06ace0 msgr2=0x7f505c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:03.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.000+0000 7f507fbb6700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f505c06ace0 0x7f505c06d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5068000f60 tx=0x7f5068007d30 comp rx=0 tx=0).stop 2023-06-23T18:37:03.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.000+0000 7f507fbb6700 1 -- 172.21.15.79:0/2450947723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50780b1270 msgr2=0x7f5078152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:03.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.000+0000 7f507fbb6700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50780b1270 0x7f5078152310 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5078070bc0 tx=0x7f5070004ad0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.000+0000 7f507fbb6700 1 -- 172.21.15.79:0/2450947723 shutdown_connections 2023-06-23T18:37:03.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.000+0000 7f507fbb6700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50780b1270 0x7f5078152310 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.000+0000 7f507fbb6700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5078071490 0x7f50781536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.001+0000 7f507fbb6700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f505c06ace0 0x7f505c06d190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.001+0000 7f507fbb6700 1 --2- 172.21.15.79:0/2450947723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50780aee30 0x7f5078151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.001+0000 7f507fbb6700 1 -- 172.21.15.79:0/2450947723 >> 172.21.15.79:0/2450947723 conn(0x7f507806d660 msgr2=0x7f50780b0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:03.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.001+0000 7f507fbb6700 1 -- 172.21.15.79:0/2450947723 shutdown_connections 2023-06-23T18:37:03.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.001+0000 7f507fbb6700 1 -- 172.21.15.79:0/2450947723 wait complete. 2023-06-23T18:37:03.009 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-23T18:37:03.018 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2023-06-23T18:37:02.709198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219812,"kb_used_data":4196,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754908,"statfs":{"total":773094113280,"available":772869025792,"internally_reserved":0,"allocated":4296704,"data_stored":2449807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.565176"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554247+0000","last_change":"2023-06-23T18:36:58.569751+0000","last_active":"2023-06-23T18:36:59.554247+0000","last_peered":"2023-06-23T18:36:59.554247+0000","last_clean":"2023-06-23T18:36:59.554247+0000","last_became_active":"2023-06-23T18:36:58.569475+0000","last_became_peered":"2023-06-23T18:36:58.569475+0000","last_unstale":"2023-06-23T18:36:59.554247+0000","last_undegraded":"2023-06-23T18:36:59.554247+0000","last_fullsized":"2023-06-23T18:36:59.554247+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:35:39.567309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.553807+0000","last_change":"2023-06-23T18:36:58.567746+0000","last_active":"2023-06-23T18:36:59.553807+0000","last_peered":"2023-06-23T18:36:59.553807+0000","last_clean":"2023-06-23T18:36:59.553807+0000","last_became_active":"2023-06-23T18:36:58.567478+0000","last_became_peered":"2023-06-23T18:36:58.567478+0000","last_unstale":"2023-06-23T18:36:59.553807+0000","last_undegraded":"2023-06-23T18:36:59.553807+0000","last_fullsized":"2023-06-23T18:36:59.553807+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:50:25.144151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573374+0000","last_change":"2023-06-23T18:36:58.559779+0000","last_active":"2023-06-23T18:37:00.573374+0000","last_peered":"2023-06-23T18:37:00.573374+0000","last_clean":"2023-06-23T18:37:00.573374+0000","last_became_active":"2023-06-23T18:36:58.559502+0000","last_became_peered":"2023-06-23T18:36:58.559502+0000","last_unstale":"2023-06-23T18:37:00.573374+0000","last_undegraded":"2023-06-23T18:37:00.573374+0000","last_fullsized":"2023-06-23T18:37:00.573374+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:15:18.176327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.408555+0000","last_change":"2023-06-23T18:36:58.563531+0000","last_active":"2023-06-23T18:37:00.408555+0000","last_peered":"2023-06-23T18:37:00.408555+0000","last_clean":"2023-06-23T18:37:00.408555+0000","last_became_active":"2023-06-23T18:36:58.563032+0000","last_became_peered":"2023-06-23T18:36:58.563032+0000","last_unstale":"2023-06-23T18:37:00.408555+0000","last_undegraded":"2023-06-23T18:37:00.408555+0000","last_fullsized":"2023-06-23T18:37:00.408555+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:06:03.243588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.408436+0000","last_change":"2023-06-23T18:36:58.562922+0000","last_active":"2023-06-23T18:37:00.408436+0000","last_peered":"2023-06-23T18:37:00.408436+0000","last_clean":"2023-06-23T18:37:00.408436+0000","last_became_active":"2023-06-23T18:36:58.562653+0000","last_became_peered":"2023-06-23T18:36:58.562653+0000","last_unstale":"2023-06-23T18:37:00.408436+0000","last_undegraded":"2023-06-23T18:37:00.408436+0000","last_fullsized":"2023-06-23T18:37:00.408436+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:44:44.227251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.328226+0000","last_change":"2023-06-23T18:36:58.571707+0000","last_active":"2023-06-23T18:37:00.328226+0000","last_peered":"2023-06-23T18:37:00.328226+0000","last_clean":"2023-06-23T18:37:00.328226+0000","last_became_active":"2023-06-23T18:36:58.571447+0000","last_became_peered":"2023-06-23T18:36:58.571447+0000","last_unstale":"2023-06-23T18:37:00.328226+0000","last_undegraded":"2023-06-23T18:37:00.328226+0000","last_fullsized":"2023-06-23T18:37:00.328226+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:34:32.116697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554660+0000","last_change":"2023-06-23T18:36:58.559278+0000","last_active":"2023-06-23T18:36:59.554660+0000","last_peered":"2023-06-23T18:36:59.554660+0000","last_clean":"2023-06-23T18:36:59.554660+0000","last_became_active":"2023-06-23T18:36:58.559000+0000","last_became_peered":"2023-06-23T18:36:58.559000+0000","last_unstale":"2023-06-23T18:36:59.554660+0000","last_undegraded":"2023-06-23T18:36:59.554660+0000","last_fullsized":"2023-06-23T18:36:59.554660+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:16:46.340440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.553289+0000","last_change":"2023-06-23T18:36:58.568604+0000","last_active":"2023-06-23T18:36:59.553289+0000","last_peered":"2023-06-23T18:36:59.553289+0000","last_clean":"2023-06-23T18:36:59.553289+0000","last_became_active":"2023-06-23T18:36:58.568362+0000","last_became_peered":"2023-06-23T18:36:58.568362+0000","last_unstale":"2023-06-23T18:36:59.553289+0000","last_undegraded":"2023-06-23T18:36:59.553289+0000","last_fullsized":"2023-06-23T18:36:59.553289+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:25:38.456175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573507+0000","last_change":"2023-06-23T18:36:58.560340+0000","last_active":"2023-06-23T18:37:00.573507+0000","last_peered":"2023-06-23T18:37:00.573507+0000","last_clean":"2023-06-23T18:37:00.573507+0000","last_became_active":"2023-06-23T18:36:58.560090+0000","last_became_peered":"2023-06-23T18:36:58.560090+0000","last_unstale":"2023-06-23T18:37:00.573507+0000","last_undegraded":"2023-06-23T18:37:00.573507+0000","last_fullsized":"2023-06-23T18:37:00.573507+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:55:26.572778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555778+0000","last_change":"2023-06-23T18:36:58.575883+0000","last_active":"2023-06-23T18:36:59.555778+0000","last_peered":"2023-06-23T18:36:59.555778+0000","last_clean":"2023-06-23T18:36:59.555778+0000","last_became_active":"2023-06-23T18:36:58.575597+0000","last_became_peered":"2023-06-23T18:36:58.575597+0000","last_unstale":"2023-06-23T18:36:59.555778+0000","last_undegraded":"2023-06-23T18:36:59.555778+0000","last_fullsized":"2023-06-23T18:36:59.555778+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:42.313495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555694+0000","last_change":"2023-06-23T18:36:58.558679+0000","last_active":"2023-06-23T18:36:59.555694+0000","last_peered":"2023-06-23T18:36:59.555694+0000","last_clean":"2023-06-23T18:36:59.555694+0000","last_became_active":"2023-06-23T18:36:58.557659+0000","last_became_peered":"2023-06-23T18:36:58.557659+0000","last_unstale":"2023-06-23T18:36:59.555694+0000","last_undegraded":"2023-06-23T18:36:59.555694+0000","last_fullsized":"2023-06-23T18:36:59.555694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:32:52.956959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573364+0000","last_change":"2023-06-23T18:36:58.563493+0000","last_active":"2023-06-23T18:37:00.573364+0000","last_peered":"2023-06-23T18:37:00.573364+0000","last_clean":"2023-06-23T18:37:00.573364+0000","last_became_active":"2023-06-23T18:36:58.563241+0000","last_became_peered":"2023-06-23T18:36:58.563241+0000","last_unstale":"2023-06-23T18:37:00.573364+0000","last_undegraded":"2023-06-23T18:37:00.573364+0000","last_fullsized":"2023-06-23T18:37:00.573364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:56.355683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555361+0000","last_change":"2023-06-23T18:36:58.575907+0000","last_active":"2023-06-23T18:36:59.555361+0000","last_peered":"2023-06-23T18:36:59.555361+0000","last_clean":"2023-06-23T18:36:59.555361+0000","last_became_active":"2023-06-23T18:36:58.575611+0000","last_became_peered":"2023-06-23T18:36:58.575611+0000","last_unstale":"2023-06-23T18:36:59.555361+0000","last_undegraded":"2023-06-23T18:36:59.555361+0000","last_fullsized":"2023-06-23T18:36:59.555361+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:14:00.864881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573866+0000","last_change":"2023-06-23T18:36:58.572666+0000","last_active":"2023-06-23T18:37:00.573866+0000","last_peered":"2023-06-23T18:37:00.573866+0000","last_clean":"2023-06-23T18:37:00.573866+0000","last_became_active":"2023-06-23T18:36:58.572175+0000","last_became_peered":"2023-06-23T18:36:58.572175+0000","last_unstale":"2023-06-23T18:37:00.573866+0000","last_undegraded":"2023-06-23T18:37:00.573866+0000","last_fullsized":"2023-06-23T18:37:00.573866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:58:25.279116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554264+0000","last_change":"2023-06-23T18:36:58.568288+0000","last_active":"2023-06-23T18:36:59.554264+0000","last_peered":"2023-06-23T18:36:59.554264+0000","last_clean":"2023-06-23T18:36:59.554264+0000","last_became_active":"2023-06-23T18:36:58.568016+0000","last_became_peered":"2023-06-23T18:36:58.568016+0000","last_unstale":"2023-06-23T18:36:59.554264+0000","last_undegraded":"2023-06-23T18:36:59.554264+0000","last_fullsized":"2023-06-23T18:36:59.554264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:41:37.829801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555149+0000","last_change":"2023-06-23T18:36:58.576270+0000","last_active":"2023-06-23T18:36:59.555149+0000","last_peered":"2023-06-23T18:36:59.555149+0000","last_clean":"2023-06-23T18:36:59.555149+0000","last_became_active":"2023-06-23T18:36:58.575967+0000","last_became_peered":"2023-06-23T18:36:58.575967+0000","last_unstale":"2023-06-23T18:36:59.555149+0000","last_undegraded":"2023-06-23T18:36:59.555149+0000","last_fullsized":"2023-06-23T18:36:59.555149+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:00:25.268981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555319+0000","last_change":"2023-06-23T18:36:58.572788+0000","last_active":"2023-06-23T18:36:59.555319+0000","last_peered":"2023-06-23T18:36:59.555319+0000","last_clean":"2023-06-23T18:36:59.555319+0000","last_became_active":"2023-06-23T18:36:58.571869+0000","last_became_peered":"2023-06-23T18:36:58.571869+0000","last_unstale":"2023-06-23T18:36:59.555319+0000","last_undegraded":"2023-06-23T18:36:59.555319+0000","last_fullsized":"2023-06-23T18:36:59.555319+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:00:17.728066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573901+0000","last_change":"2023-06-23T18:36:58.567330+0000","last_active":"2023-06-23T18:37:00.573901+0000","last_peered":"2023-06-23T18:37:00.573901+0000","last_clean":"2023-06-23T18:37:00.573901+0000","last_became_active":"2023-06-23T18:36:58.566657+0000","last_became_peered":"2023-06-23T18:36:58.566657+0000","last_unstale":"2023-06-23T18:37:00.573901+0000","last_undegraded":"2023-06-23T18:37:00.573901+0000","last_fullsized":"2023-06-23T18:37:00.573901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:53:25.751012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.408615+0000","last_change":"2023-06-23T18:36:58.564655+0000","last_active":"2023-06-23T18:37:00.408615+0000","last_peered":"2023-06-23T18:37:00.408615+0000","last_clean":"2023-06-23T18:37:00.408615+0000","last_became_active":"2023-06-23T18:36:58.564404+0000","last_became_peered":"2023-06-23T18:36:58.564404+0000","last_unstale":"2023-06-23T18:37:00.408615+0000","last_undegraded":"2023-06-23T18:37:00.408615+0000","last_fullsized":"2023-06-23T18:37:00.408615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:14:23.687288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573937+0000","last_change":"2023-06-23T18:36:58.574903+0000","last_active":"2023-06-23T18:37:00.573937+0000","last_peered":"2023-06-23T18:37:00.573937+0000","last_clean":"2023-06-23T18:37:00.573937+0000","last_became_active":"2023-06-23T18:36:58.574609+0000","last_became_peered":"2023-06-23T18:36:58.574609+0000","last_unstale":"2023-06-23T18:37:00.573937+0000","last_undegraded":"2023-06-23T18:37:00.573937+0000","last_fullsized":"2023-06-23T18:37:00.573937+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:51:29.811119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555267+0000","last_change":"2023-06-23T18:36:58.563611+0000","last_active":"2023-06-23T18:36:59.555267+0000","last_peered":"2023-06-23T18:36:59.555267+0000","last_clean":"2023-06-23T18:36:59.555267+0000","last_became_active":"2023-06-23T18:36:58.563314+0000","last_became_peered":"2023-06-23T18:36:58.563314+0000","last_unstale":"2023-06-23T18:36:59.555267+0000","last_undegraded":"2023-06-23T18:36:59.555267+0000","last_fullsized":"2023-06-23T18:36:59.555267+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:00:26.131269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573995+0000","last_change":"2023-06-23T18:36:58.565224+0000","last_active":"2023-06-23T18:37:00.573995+0000","last_peered":"2023-06-23T18:37:00.573995+0000","last_clean":"2023-06-23T18:37:00.573995+0000","last_became_active":"2023-06-23T18:36:58.564950+0000","last_became_peered":"2023-06-23T18:36:58.564950+0000","last_unstale":"2023-06-23T18:37:00.573995+0000","last_undegraded":"2023-06-23T18:37:00.573995+0000","last_fullsized":"2023-06-23T18:37:00.573995+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:51:08.614432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554264+0000","last_change":"2023-06-23T18:36:58.577420+0000","last_active":"2023-06-23T18:36:59.554264+0000","last_peered":"2023-06-23T18:36:59.554264+0000","last_clean":"2023-06-23T18:36:59.554264+0000","last_became_active":"2023-06-23T18:36:58.577096+0000","last_became_peered":"2023-06-23T18:36:58.577096+0000","last_unstale":"2023-06-23T18:36:59.554264+0000","last_undegraded":"2023-06-23T18:36:59.554264+0000","last_fullsized":"2023-06-23T18:36:59.554264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:12:26.932265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.408362+0000","last_change":"2023-06-23T18:36:58.575761+0000","last_active":"2023-06-23T18:37:00.408362+0000","last_peered":"2023-06-23T18:37:00.408362+0000","last_clean":"2023-06-23T18:37:00.408362+0000","last_became_active":"2023-06-23T18:36:58.575542+0000","last_became_peered":"2023-06-23T18:36:58.575542+0000","last_unstale":"2023-06-23T18:37:00.408362+0000","last_undegraded":"2023-06-23T18:37:00.408362+0000","last_fullsized":"2023-06-23T18:37:00.408362+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:09:41.076244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.574362+0000","last_change":"2023-06-23T18:36:58.569864+0000","last_active":"2023-06-23T18:37:00.574362+0000","last_peered":"2023-06-23T18:37:00.574362+0000","last_clean":"2023-06-23T18:37:00.574362+0000","last_became_active":"2023-06-23T18:36:58.569547+0000","last_became_peered":"2023-06-23T18:36:58.569547+0000","last_unstale":"2023-06-23T18:37:00.574362+0000","last_undegraded":"2023-06-23T18:37:00.574362+0000","last_fullsized":"2023-06-23T18:37:00.574362+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:58:03.734338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.328431+0000","last_change":"2023-06-23T18:36:58.563653+0000","last_active":"2023-06-23T18:37:00.328431+0000","last_peered":"2023-06-23T18:37:00.328431+0000","last_clean":"2023-06-23T18:37:00.328431+0000","last_became_active":"2023-06-23T18:36:58.563445+0000","last_became_peered":"2023-06-23T18:36:58.563445+0000","last_unstale":"2023-06-23T18:37:00.328431+0000","last_undegraded":"2023-06-23T18:37:00.328431+0000","last_fullsized":"2023-06-23T18:37:00.328431+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:17.965706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573992+0000","last_change":"2023-06-23T18:36:58.567428+0000","last_active":"2023-06-23T18:37:00.573992+0000","last_peered":"2023-06-23T18:37:00.573992+0000","last_clean":"2023-06-23T18:37:00.573992+0000","last_became_active":"2023-06-23T18:36:58.566143+0000","last_became_peered":"2023-06-23T18:36:58.566143+0000","last_unstale":"2023-06-23T18:37:00.573992+0000","last_undegraded":"2023-06-23T18:37:00.573992+0000","last_fullsized":"2023-06-23T18:37:00.573992+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:44:06.064989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555176+0000","last_change":"2023-06-23T18:36:58.566342+0000","last_active":"2023-06-23T18:36:59.555176+0000","last_peered":"2023-06-23T18:36:59.555176+0000","last_clean":"2023-06-23T18:36:59.555176+0000","last_became_active":"2023-06-23T18:36:58.565620+0000","last_became_peered":"2023-06-23T18:36:58.565620+0000","last_unstale":"2023-06-23T18:36:59.555176+0000","last_undegraded":"2023-06-23T18:36:59.555176+0000","last_fullsized":"2023-06-23T18:36:59.555176+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:47:14.699160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.574225+0000","last_change":"2023-06-23T18:36:58.566935+0000","last_active":"2023-06-23T18:37:00.574225+0000","last_peered":"2023-06-23T18:37:00.574225+0000","last_clean":"2023-06-23T18:37:00.574225+0000","last_became_active":"2023-06-23T18:36:58.565900+0000","last_became_peered":"2023-06-23T18:36:58.565900+0000","last_unstale":"2023-06-23T18:37:00.574225+0000","last_undegraded":"2023-06-23T18:37:00.574225+0000","last_fullsized":"2023-06-23T18:37:00.574225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:34:32.116697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573861+0000","last_change":"2023-06-23T18:36:58.574453+0000","last_active":"2023-06-23T18:37:00.573861+0000","last_peered":"2023-06-23T18:37:00.573861+0000","last_clean":"2023-06-23T18:37:00.573861+0000","last_became_active":"2023-06-23T18:36:58.574024+0000","last_became_peered":"2023-06-23T18:36:58.574024+0000","last_unstale":"2023-06-23T18:37:00.573861+0000","last_undegraded":"2023-06-23T18:37:00.573861+0000","last_fullsized":"2023-06-23T18:37:00.573861+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:42:35.013156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554202+0000","last_change":"2023-06-23T18:36:58.577425+0000","last_active":"2023-06-23T18:36:59.554202+0000","last_peered":"2023-06-23T18:36:59.554202+0000","last_clean":"2023-06-23T18:36:59.554202+0000","last_became_active":"2023-06-23T18:36:58.576719+0000","last_became_peered":"2023-06-23T18:36:58.576719+0000","last_unstale":"2023-06-23T18:36:59.554202+0000","last_undegraded":"2023-06-23T18:36:59.554202+0000","last_fullsized":"2023-06-23T18:36:59.554202+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:40:58.278104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.328161+0000","last_change":"2023-06-23T18:36:58.565774+0000","last_active":"2023-06-23T18:37:00.328161+0000","last_peered":"2023-06-23T18:37:00.328161+0000","last_clean":"2023-06-23T18:37:00.328161+0000","last_became_active":"2023-06-23T18:36:58.565482+0000","last_became_peered":"2023-06-23T18:36:58.565482+0000","last_unstale":"2023-06-23T18:37:00.328161+0000","last_undegraded":"2023-06-23T18:37:00.328161+0000","last_fullsized":"2023-06-23T18:37:00.328161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:09:24.969467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554985+0000","last_change":"2023-06-23T18:36:58.576221+0000","last_active":"2023-06-23T18:36:59.554985+0000","last_peered":"2023-06-23T18:36:59.554985+0000","last_clean":"2023-06-23T18:36:59.554985+0000","last_became_active":"2023-06-23T18:36:58.575913+0000","last_became_peered":"2023-06-23T18:36:58.575913+0000","last_unstale":"2023-06-23T18:36:59.554985+0000","last_undegraded":"2023-06-23T18:36:59.554985+0000","last_fullsized":"2023-06-23T18:36:59.554985+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:15:13.113268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.574527+0000","last_change":"2023-06-23T18:36:58.167599+0000","last_active":"2023-06-23T18:37:00.574527+0000","last_peered":"2023-06-23T18:37:00.574527+0000","last_clean":"2023-06-23T18:37:00.574527+0000","last_became_active":"2023-06-23T18:36:58.167365+0000","last_became_peered":"2023-06-23T18:36:58.167365+0000","last_unstale":"2023-06-23T18:37:00.574527+0000","last_undegraded":"2023-06-23T18:37:00.574527+0000","last_fullsized":"2023-06-23T18:37:00.574527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:12:46.494019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.328503+0000","last_change":"2023-06-23T18:36:58.568820+0000","last_active":"2023-06-23T18:37:00.328503+0000","last_peered":"2023-06-23T18:37:00.328503+0000","last_clean":"2023-06-23T18:37:00.328503+0000","last_became_active":"2023-06-23T18:36:58.568523+0000","last_became_peered":"2023-06-23T18:36:58.568523+0000","last_unstale":"2023-06-23T18:37:00.328503+0000","last_undegraded":"2023-06-23T18:37:00.328503+0000","last_fullsized":"2023-06-23T18:37:00.328503+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:27:22.872319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573492+0000","last_change":"2023-06-23T18:36:58.160047+0000","last_active":"2023-06-23T18:37:00.573492+0000","last_peered":"2023-06-23T18:37:00.573492+0000","last_clean":"2023-06-23T18:37:00.573492+0000","last_became_active":"2023-06-23T18:36:58.159751+0000","last_became_peered":"2023-06-23T18:36:58.159751+0000","last_unstale":"2023-06-23T18:37:00.573492+0000","last_undegraded":"2023-06-23T18:37:00.573492+0000","last_fullsized":"2023-06-23T18:37:00.573492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:23:26.338175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.574509+0000","last_change":"2023-06-23T18:36:58.573985+0000","last_active":"2023-06-23T18:37:00.574509+0000","last_peered":"2023-06-23T18:37:00.574509+0000","last_clean":"2023-06-23T18:37:00.574509+0000","last_became_active":"2023-06-23T18:36:58.573501+0000","last_became_peered":"2023-06-23T18:36:58.573501+0000","last_unstale":"2023-06-23T18:37:00.574509+0000","last_undegraded":"2023-06-23T18:37:00.574509+0000","last_fullsized":"2023-06-23T18:37:00.574509+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:13:12.427528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.328620+0000","last_change":"2023-06-23T18:36:58.162491+0000","last_active":"2023-06-23T18:37:00.328620+0000","last_peered":"2023-06-23T18:37:00.328620+0000","last_clean":"2023-06-23T18:37:00.328620+0000","last_became_active":"2023-06-23T18:36:58.162204+0000","last_became_peered":"2023-06-23T18:36:58.162204+0000","last_unstale":"2023-06-23T18:37:00.328620+0000","last_undegraded":"2023-06-23T18:37:00.328620+0000","last_fullsized":"2023-06-23T18:37:00.328620+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:13:05.558097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554122+0000","last_change":"2023-06-23T18:36:58.570017+0000","last_active":"2023-06-23T18:36:59.554122+0000","last_peered":"2023-06-23T18:36:59.554122+0000","last_clean":"2023-06-23T18:36:59.554122+0000","last_became_active":"2023-06-23T18:36:58.569756+0000","last_became_peered":"2023-06-23T18:36:58.569756+0000","last_unstale":"2023-06-23T18:36:59.554122+0000","last_undegraded":"2023-06-23T18:36:59.554122+0000","last_fullsized":"2023-06-23T18:36:59.554122+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:58:55.136134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.574233+0000","last_change":"2023-06-23T18:36:58.162239+0000","last_active":"2023-06-23T18:37:00.574233+0000","last_peered":"2023-06-23T18:37:00.574233+0000","last_clean":"2023-06-23T18:37:00.574233+0000","last_became_active":"2023-06-23T18:36:58.161972+0000","last_became_peered":"2023-06-23T18:36:58.161972+0000","last_unstale":"2023-06-23T18:37:00.574233+0000","last_undegraded":"2023-06-23T18:37:00.574233+0000","last_fullsized":"2023-06-23T18:37:00.574233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:28:07.361249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573076+0000","last_change":"2023-06-23T18:36:58.568094+0000","last_active":"2023-06-23T18:37:00.573076+0000","last_peered":"2023-06-23T18:37:00.573076+0000","last_clean":"2023-06-23T18:37:00.573076+0000","last_became_active":"2023-06-23T18:36:58.567832+0000","last_became_peered":"2023-06-23T18:36:58.567832+0000","last_unstale":"2023-06-23T18:37:00.573076+0000","last_undegraded":"2023-06-23T18:37:00.573076+0000","last_fullsized":"2023-06-23T18:37:00.573076+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:13:06.564431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555115+0000","last_change":"2023-06-23T18:36:58.163486+0000","last_active":"2023-06-23T18:36:59.555115+0000","last_peered":"2023-06-23T18:36:59.555115+0000","last_clean":"2023-06-23T18:36:59.555115+0000","last_became_active":"2023-06-23T18:36:58.163067+0000","last_became_peered":"2023-06-23T18:36:58.163067+0000","last_unstale":"2023-06-23T18:36:59.555115+0000","last_undegraded":"2023-06-23T18:36:59.555115+0000","last_fullsized":"2023-06-23T18:36:59.555115+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:21:35.780609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554505+0000","last_change":"2023-06-23T18:36:58.562688+0000","last_active":"2023-06-23T18:36:59.554505+0000","last_peered":"2023-06-23T18:36:59.554505+0000","last_clean":"2023-06-23T18:36:59.554505+0000","last_became_active":"2023-06-23T18:36:58.562455+0000","last_became_peered":"2023-06-23T18:36:58.562455+0000","last_unstale":"2023-06-23T18:36:59.554505+0000","last_undegraded":"2023-06-23T18:36:59.554505+0000","last_fullsized":"2023-06-23T18:36:59.554505+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:36:50.602882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555407+0000","last_change":"2023-06-23T18:36:58.165351+0000","last_active":"2023-06-23T18:36:59.555407+0000","last_peered":"2023-06-23T18:36:59.555407+0000","last_clean":"2023-06-23T18:36:59.555407+0000","last_became_active":"2023-06-23T18:36:58.164998+0000","last_became_peered":"2023-06-23T18:36:58.164998+0000","last_unstale":"2023-06-23T18:36:59.555407+0000","last_undegraded":"2023-06-23T18:36:59.555407+0000","last_fullsized":"2023-06-23T18:36:59.555407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:36:31.667662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.574010+0000","last_change":"2023-06-23T18:36:58.558795+0000","last_active":"2023-06-23T18:37:00.574010+0000","last_peered":"2023-06-23T18:37:00.574010+0000","last_clean":"2023-06-23T18:37:00.574010+0000","last_became_active":"2023-06-23T18:36:58.557402+0000","last_became_peered":"2023-06-23T18:36:58.557402+0000","last_unstale":"2023-06-23T18:37:00.574010+0000","last_undegraded":"2023-06-23T18:37:00.574010+0000","last_fullsized":"2023-06-23T18:37:00.574010+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:36:00.594979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.328269+0000","last_change":"2023-06-23T18:36:58.168210+0000","last_active":"2023-06-23T18:37:00.328269+0000","last_peered":"2023-06-23T18:37:00.328269+0000","last_clean":"2023-06-23T18:37:00.328269+0000","last_became_active":"2023-06-23T18:36:58.167925+0000","last_became_peered":"2023-06-23T18:36:58.167925+0000","last_unstale":"2023-06-23T18:37:00.328269+0000","last_undegraded":"2023-06-23T18:37:00.328269+0000","last_fullsized":"2023-06-23T18:37:00.328269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:12:46.494019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.553650+0000","last_change":"2023-06-23T18:36:58.570491+0000","last_active":"2023-06-23T18:36:59.553650+0000","last_peered":"2023-06-23T18:36:59.553650+0000","last_clean":"2023-06-23T18:36:59.553650+0000","last_became_active":"2023-06-23T18:36:58.570212+0000","last_became_peered":"2023-06-23T18:36:58.570212+0000","last_unstale":"2023-06-23T18:36:59.553650+0000","last_undegraded":"2023-06-23T18:36:59.553650+0000","last_fullsized":"2023-06-23T18:36:59.553650+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:12:26.932265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573472+0000","last_change":"2023-06-23T18:36:58.164326+0000","last_active":"2023-06-23T18:37:00.573472+0000","last_peered":"2023-06-23T18:37:00.573472+0000","last_clean":"2023-06-23T18:37:00.573472+0000","last_became_active":"2023-06-23T18:36:58.164005+0000","last_became_peered":"2023-06-23T18:36:58.164005+0000","last_unstale":"2023-06-23T18:37:00.573472+0000","last_undegraded":"2023-06-23T18:37:00.573472+0000","last_fullsized":"2023-06-23T18:37:00.573472+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:35:13.445488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.408659+0000","last_change":"2023-06-23T18:36:58.563794+0000","last_active":"2023-06-23T18:37:00.408659+0000","last_peered":"2023-06-23T18:37:00.408659+0000","last_clean":"2023-06-23T18:37:00.408659+0000","last_became_active":"2023-06-23T18:36:58.563293+0000","last_became_peered":"2023-06-23T18:36:58.563293+0000","last_unstale":"2023-06-23T18:37:00.408659+0000","last_undegraded":"2023-06-23T18:37:00.408659+0000","last_fullsized":"2023-06-23T18:37:00.408659+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:45:19.436190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554182+0000","last_change":"2023-06-23T18:36:58.166945+0000","last_active":"2023-06-23T18:36:59.554182+0000","last_peered":"2023-06-23T18:36:59.554182+0000","last_clean":"2023-06-23T18:36:59.554182+0000","last_became_active":"2023-06-23T18:36:58.166652+0000","last_became_peered":"2023-06-23T18:36:58.166652+0000","last_unstale":"2023-06-23T18:36:59.554182+0000","last_undegraded":"2023-06-23T18:36:59.554182+0000","last_fullsized":"2023-06-23T18:36:59.554182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:15:12.106934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554958+0000","last_change":"2023-06-23T18:36:58.571282+0000","last_active":"2023-06-23T18:36:59.554958+0000","last_peered":"2023-06-23T18:36:59.554958+0000","last_clean":"2023-06-23T18:36:59.554958+0000","last_became_active":"2023-06-23T18:36:58.570982+0000","last_became_peered":"2023-06-23T18:36:58.570982+0000","last_unstale":"2023-06-23T18:36:59.554958+0000","last_undegraded":"2023-06-23T18:36:59.554958+0000","last_fullsized":"2023-06-23T18:36:59.554958+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:17:11.138327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555743+0000","last_change":"2023-06-23T18:36:58.160492+0000","last_active":"2023-06-23T18:36:59.555743+0000","last_peered":"2023-06-23T18:36:59.555743+0000","last_clean":"2023-06-23T18:36:59.555743+0000","last_became_active":"2023-06-23T18:36:58.160210+0000","last_became_peered":"2023-06-23T18:36:58.160210+0000","last_unstale":"2023-06-23T18:36:59.555743+0000","last_undegraded":"2023-06-23T18:36:59.555743+0000","last_fullsized":"2023-06-23T18:36:59.555743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:11:33.831427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573485+0000","last_change":"2023-06-23T18:36:58.571100+0000","last_active":"2023-06-23T18:37:00.573485+0000","last_peered":"2023-06-23T18:37:00.573485+0000","last_clean":"2023-06-23T18:37:00.573485+0000","last_became_active":"2023-06-23T18:36:58.570798+0000","last_became_peered":"2023-06-23T18:36:58.570798+0000","last_unstale":"2023-06-23T18:37:00.573485+0000","last_undegraded":"2023-06-23T18:37:00.573485+0000","last_fullsized":"2023-06-23T18:37:00.573485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:23:26.080405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555732+0000","last_change":"2023-06-23T18:36:58.166980+0000","last_active":"2023-06-23T18:36:59.555732+0000","last_peered":"2023-06-23T18:36:59.555732+0000","last_clean":"2023-06-23T18:36:59.555732+0000","last_became_active":"2023-06-23T18:36:58.166696+0000","last_became_peered":"2023-06-23T18:36:58.166696+0000","last_unstale":"2023-06-23T18:36:59.555732+0000","last_undegraded":"2023-06-23T18:36:59.555732+0000","last_fullsized":"2023-06-23T18:36:59.555732+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:13:56.890634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.574480+0000","last_change":"2023-06-23T18:36:58.563763+0000","last_active":"2023-06-23T18:37:00.574480+0000","last_peered":"2023-06-23T18:37:00.574480+0000","last_clean":"2023-06-23T18:37:00.574480+0000","last_became_active":"2023-06-23T18:36:58.563463+0000","last_became_peered":"2023-06-23T18:36:58.563463+0000","last_unstale":"2023-06-23T18:37:00.574480+0000","last_undegraded":"2023-06-23T18:37:00.574480+0000","last_fullsized":"2023-06-23T18:37:00.574480+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:42.285459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.553868+0000","last_change":"2023-06-23T18:36:58.158355+0000","last_active":"2023-06-23T18:36:59.553868+0000","last_peered":"2023-06-23T18:36:59.553868+0000","last_clean":"2023-06-23T18:36:59.553868+0000","last_became_active":"2023-06-23T18:36:58.158085+0000","last_became_peered":"2023-06-23T18:36:58.158085+0000","last_unstale":"2023-06-23T18:36:59.553868+0000","last_undegraded":"2023-06-23T18:36:59.553868+0000","last_fullsized":"2023-06-23T18:36:59.553868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:28:44.170326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573240+0000","last_change":"2023-06-23T18:36:58.559803+0000","last_active":"2023-06-23T18:37:00.573240+0000","last_peered":"2023-06-23T18:37:00.573240+0000","last_clean":"2023-06-23T18:37:00.573240+0000","last_became_active":"2023-06-23T18:36:58.559509+0000","last_became_peered":"2023-06-23T18:36:58.559509+0000","last_unstale":"2023-06-23T18:37:00.573240+0000","last_undegraded":"2023-06-23T18:37:00.573240+0000","last_fullsized":"2023-06-23T18:37:00.573240+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:59:00.120066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.553788+0000","last_change":"2023-06-23T18:36:58.159534+0000","last_active":"2023-06-23T18:36:59.553788+0000","last_peered":"2023-06-23T18:36:59.553788+0000","last_clean":"2023-06-23T18:36:59.553788+0000","last_became_active":"2023-06-23T18:36:58.158983+0000","last_became_peered":"2023-06-23T18:36:58.158983+0000","last_unstale":"2023-06-23T18:36:59.553788+0000","last_undegraded":"2023-06-23T18:36:59.553788+0000","last_fullsized":"2023-06-23T18:36:59.553788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:11:51.560697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.553383+0000","last_change":"2023-06-23T18:36:58.569747+0000","last_active":"2023-06-23T18:36:59.553383+0000","last_peered":"2023-06-23T18:36:59.553383+0000","last_clean":"2023-06-23T18:36:59.553383+0000","last_became_active":"2023-06-23T18:36:58.569527+0000","last_became_peered":"2023-06-23T18:36:58.569527+0000","last_unstale":"2023-06-23T18:36:59.553383+0000","last_undegraded":"2023-06-23T18:36:59.553383+0000","last_fullsized":"2023-06-23T18:36:59.553383+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:38:22.277113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.328410+0000","last_change":"2023-06-23T18:36:58.168262+0000","last_active":"2023-06-23T18:37:00.328410+0000","last_peered":"2023-06-23T18:37:00.328410+0000","last_clean":"2023-06-23T18:37:00.328410+0000","last_became_active":"2023-06-23T18:36:58.167972+0000","last_became_peered":"2023-06-23T18:36:58.167972+0000","last_unstale":"2023-06-23T18:37:00.328410+0000","last_undegraded":"2023-06-23T18:37:00.328410+0000","last_fullsized":"2023-06-23T18:37:00.328410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:06:37.101101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554307+0000","last_change":"2023-06-23T18:36:58.571434+0000","last_active":"2023-06-23T18:36:59.554307+0000","last_peered":"2023-06-23T18:36:59.554307+0000","last_clean":"2023-06-23T18:36:59.554307+0000","last_became_active":"2023-06-23T18:36:58.571013+0000","last_became_peered":"2023-06-23T18:36:58.571013+0000","last_unstale":"2023-06-23T18:36:59.554307+0000","last_undegraded":"2023-06-23T18:36:59.554307+0000","last_fullsized":"2023-06-23T18:36:59.554307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:14:30.697747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.574049+0000","last_change":"2023-06-23T18:36:58.167714+0000","last_active":"2023-06-23T18:37:00.574049+0000","last_peered":"2023-06-23T18:37:00.574049+0000","last_clean":"2023-06-23T18:37:00.574049+0000","last_became_active":"2023-06-23T18:36:58.167415+0000","last_became_peered":"2023-06-23T18:36:58.167415+0000","last_unstale":"2023-06-23T18:37:00.574049+0000","last_undegraded":"2023-06-23T18:37:00.574049+0000","last_fullsized":"2023-06-23T18:37:00.574049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:08:12.656946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.553402+0000","last_change":"2023-06-23T18:36:58.571007+0000","last_active":"2023-06-23T18:36:59.553402+0000","last_peered":"2023-06-23T18:36:59.553402+0000","last_clean":"2023-06-23T18:36:59.553402+0000","last_became_active":"2023-06-23T18:36:58.570666+0000","last_became_peered":"2023-06-23T18:36:58.570666+0000","last_unstale":"2023-06-23T18:36:59.553402+0000","last_undegraded":"2023-06-23T18:36:59.553402+0000","last_fullsized":"2023-06-23T18:36:59.553402+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:40:58.278104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"33'8","reported_seq":21,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554835+0000","last_change":"2023-06-23T18:36:58.165076+0000","last_active":"2023-06-23T18:36:59.554835+0000","last_peered":"2023-06-23T18:36:59.554835+0000","last_clean":"2023-06-23T18:36:59.554835+0000","last_became_active":"2023-06-23T18:36:58.164779+0000","last_became_peered":"2023-06-23T18:36:58.164779+0000","last_unstale":"2023-06-23T18:36:59.554835+0000","last_undegraded":"2023-06-23T18:36:59.554835+0000","last_fullsized":"2023-06-23T18:36:59.554835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:28:44.170326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.553832+0000","last_change":"2023-06-23T18:36:58.567754+0000","last_active":"2023-06-23T18:36:59.553832+0000","last_peered":"2023-06-23T18:36:59.553832+0000","last_clean":"2023-06-23T18:36:59.553832+0000","last_became_active":"2023-06-23T18:36:58.567515+0000","last_became_peered":"2023-06-23T18:36:58.567515+0000","last_unstale":"2023-06-23T18:36:59.553832+0000","last_undegraded":"2023-06-23T18:36:59.553832+0000","last_fullsized":"2023-06-23T18:36:59.553832+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:12:25.935402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555892+0000","last_change":"2023-06-23T18:36:58.161859+0000","last_active":"2023-06-23T18:36:59.555892+0000","last_peered":"2023-06-23T18:36:59.555892+0000","last_clean":"2023-06-23T18:36:59.555892+0000","last_became_active":"2023-06-23T18:36:58.161604+0000","last_became_peered":"2023-06-23T18:36:58.161604+0000","last_unstale":"2023-06-23T18:36:59.555892+0000","last_undegraded":"2023-06-23T18:36:59.555892+0000","last_fullsized":"2023-06-23T18:36:59.555892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:22:01.197428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555548+0000","last_change":"2023-06-23T18:36:58.563535+0000","last_active":"2023-06-23T18:36:59.555548+0000","last_peered":"2023-06-23T18:36:59.555548+0000","last_clean":"2023-06-23T18:36:59.555548+0000","last_became_active":"2023-06-23T18:36:58.563238+0000","last_became_peered":"2023-06-23T18:36:58.563238+0000","last_unstale":"2023-06-23T18:36:59.555548+0000","last_undegraded":"2023-06-23T18:36:59.555548+0000","last_fullsized":"2023-06-23T18:36:59.555548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:52:52.306427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.574609+0000","last_change":"2023-06-23T18:36:58.166891+0000","last_active":"2023-06-23T18:37:00.574609+0000","last_peered":"2023-06-23T18:37:00.574609+0000","last_clean":"2023-06-23T18:37:00.574609+0000","last_became_active":"2023-06-23T18:36:58.166588+0000","last_became_peered":"2023-06-23T18:36:58.166588+0000","last_unstale":"2023-06-23T18:37:00.574609+0000","last_undegraded":"2023-06-23T18:37:00.574609+0000","last_fullsized":"2023-06-23T18:37:00.574609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:41.413412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.574531+0000","last_change":"2023-06-23T18:36:58.561200+0000","last_active":"2023-06-23T18:37:00.574531+0000","last_peered":"2023-06-23T18:37:00.574531+0000","last_clean":"2023-06-23T18:37:00.574531+0000","last_became_active":"2023-06-23T18:36:58.560940+0000","last_became_peered":"2023-06-23T18:36:58.560940+0000","last_unstale":"2023-06-23T18:37:00.574531+0000","last_undegraded":"2023-06-23T18:37:00.574531+0000","last_fullsized":"2023-06-23T18:37:00.574531+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:09:08.897994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.408667+0000","last_change":"2023-06-23T18:36:58.164946+0000","last_active":"2023-06-23T18:37:00.408667+0000","last_peered":"2023-06-23T18:37:00.408667+0000","last_clean":"2023-06-23T18:37:00.408667+0000","last_became_active":"2023-06-23T18:36:58.164668+0000","last_became_peered":"2023-06-23T18:36:58.164668+0000","last_unstale":"2023-06-23T18:37:00.408667+0000","last_undegraded":"2023-06-23T18:37:00.408667+0000","last_fullsized":"2023-06-23T18:37:00.408667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:59:10.502381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.408696+0000","last_change":"2023-06-23T18:36:58.572557+0000","last_active":"2023-06-23T18:37:00.408696+0000","last_peered":"2023-06-23T18:37:00.408696+0000","last_clean":"2023-06-23T18:37:00.408696+0000","last_became_active":"2023-06-23T18:36:58.572264+0000","last_became_peered":"2023-06-23T18:36:58.572264+0000","last_unstale":"2023-06-23T18:37:00.408696+0000","last_undegraded":"2023-06-23T18:37:00.408696+0000","last_fullsized":"2023-06-23T18:37:00.408696+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:55:03.424773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.328267+0000","last_change":"2023-06-23T18:36:58.166706+0000","last_active":"2023-06-23T18:37:00.328267+0000","last_peered":"2023-06-23T18:37:00.328267+0000","last_clean":"2023-06-23T18:37:00.328267+0000","last_became_active":"2023-06-23T18:36:58.166296+0000","last_became_peered":"2023-06-23T18:36:58.166296+0000","last_unstale":"2023-06-23T18:37:00.328267+0000","last_undegraded":"2023-06-23T18:37:00.328267+0000","last_fullsized":"2023-06-23T18:37:00.328267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:58:02.728004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554673+0000","last_change":"2023-06-23T18:36:58.571268+0000","last_active":"2023-06-23T18:36:59.554673+0000","last_peered":"2023-06-23T18:36:59.554673+0000","last_clean":"2023-06-23T18:36:59.554673+0000","last_became_active":"2023-06-23T18:36:58.570957+0000","last_became_peered":"2023-06-23T18:36:58.570957+0000","last_unstale":"2023-06-23T18:36:59.554673+0000","last_undegraded":"2023-06-23T18:36:59.554673+0000","last_fullsized":"2023-06-23T18:36:59.554673+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:12:25.935402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.408655+0000","last_change":"2023-06-23T18:36:58.162354+0000","last_active":"2023-06-23T18:37:00.408655+0000","last_peered":"2023-06-23T18:37:00.408655+0000","last_clean":"2023-06-23T18:37:00.408655+0000","last_became_active":"2023-06-23T18:36:58.162086+0000","last_became_peered":"2023-06-23T18:36:58.162086+0000","last_unstale":"2023-06-23T18:37:00.408655+0000","last_undegraded":"2023-06-23T18:37:00.408655+0000","last_fullsized":"2023-06-23T18:37:00.408655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:20:18.531568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573140+0000","last_change":"2023-06-23T18:36:58.560560+0000","last_active":"2023-06-23T18:37:00.573140+0000","last_peered":"2023-06-23T18:37:00.573140+0000","last_clean":"2023-06-23T18:37:00.573140+0000","last_became_active":"2023-06-23T18:36:58.560288+0000","last_became_peered":"2023-06-23T18:36:58.560288+0000","last_unstale":"2023-06-23T18:37:00.573140+0000","last_undegraded":"2023-06-23T18:37:00.573140+0000","last_fullsized":"2023-06-23T18:37:00.573140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:07:52.432744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.408443+0000","last_change":"2023-06-23T18:36:58.159764+0000","last_active":"2023-06-23T18:37:00.408443+0000","last_peered":"2023-06-23T18:37:00.408443+0000","last_clean":"2023-06-23T18:37:00.408443+0000","last_became_active":"2023-06-23T18:36:58.159511+0000","last_became_peered":"2023-06-23T18:36:58.159511+0000","last_unstale":"2023-06-23T18:37:00.408443+0000","last_undegraded":"2023-06-23T18:37:00.408443+0000","last_fullsized":"2023-06-23T18:37:00.408443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:56:10.614051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555040+0000","last_change":"2023-06-23T18:36:58.567155+0000","last_active":"2023-06-23T18:36:59.555040+0000","last_peered":"2023-06-23T18:36:59.555040+0000","last_clean":"2023-06-23T18:36:59.555040+0000","last_became_active":"2023-06-23T18:36:58.566855+0000","last_became_peered":"2023-06-23T18:36:58.566855+0000","last_unstale":"2023-06-23T18:36:59.555040+0000","last_undegraded":"2023-06-23T18:36:59.555040+0000","last_fullsized":"2023-06-23T18:36:59.555040+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:31:18.007762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555442+0000","last_change":"2023-06-23T18:36:58.163955+0000","last_active":"2023-06-23T18:36:59.555442+0000","last_peered":"2023-06-23T18:36:59.555442+0000","last_clean":"2023-06-23T18:36:59.555442+0000","last_became_active":"2023-06-23T18:36:58.163634+0000","last_became_peered":"2023-06-23T18:36:58.163634+0000","last_unstale":"2023-06-23T18:36:59.555442+0000","last_undegraded":"2023-06-23T18:36:59.555442+0000","last_fullsized":"2023-06-23T18:36:59.555442+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:11:51.560697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.553651+0000","last_change":"2023-06-23T18:36:58.571583+0000","last_active":"2023-06-23T18:36:59.553651+0000","last_peered":"2023-06-23T18:36:59.553651+0000","last_clean":"2023-06-23T18:36:59.553651+0000","last_became_active":"2023-06-23T18:36:58.571294+0000","last_became_peered":"2023-06-23T18:36:58.571294+0000","last_unstale":"2023-06-23T18:36:59.553651+0000","last_undegraded":"2023-06-23T18:36:59.553651+0000","last_fullsized":"2023-06-23T18:36:59.553651+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:14:30.697747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.408364+0000","last_change":"2023-06-23T18:36:58.167580+0000","last_active":"2023-06-23T18:37:00.408364+0000","last_peered":"2023-06-23T18:37:00.408364+0000","last_clean":"2023-06-23T18:37:00.408364+0000","last_became_active":"2023-06-23T18:36:58.167311+0000","last_became_peered":"2023-06-23T18:36:58.167311+0000","last_unstale":"2023-06-23T18:37:00.408364+0000","last_undegraded":"2023-06-23T18:37:00.408364+0000","last_fullsized":"2023-06-23T18:37:00.408364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:53:13.890363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.574258+0000","last_change":"2023-06-23T18:36:58.572450+0000","last_active":"2023-06-23T18:37:00.574258+0000","last_peered":"2023-06-23T18:37:00.574258+0000","last_clean":"2023-06-23T18:37:00.574258+0000","last_became_active":"2023-06-23T18:36:58.571799+0000","last_became_peered":"2023-06-23T18:36:58.571799+0000","last_unstale":"2023-06-23T18:37:00.574258+0000","last_undegraded":"2023-06-23T18:37:00.574258+0000","last_fullsized":"2023-06-23T18:37:00.574258+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:33:01.973100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573545+0000","last_change":"2023-06-23T18:36:58.161438+0000","last_active":"2023-06-23T18:37:00.573545+0000","last_peered":"2023-06-23T18:37:00.573545+0000","last_clean":"2023-06-23T18:37:00.573545+0000","last_became_active":"2023-06-23T18:36:58.161114+0000","last_became_peered":"2023-06-23T18:36:58.161114+0000","last_unstale":"2023-06-23T18:37:00.573545+0000","last_undegraded":"2023-06-23T18:37:00.573545+0000","last_fullsized":"2023-06-23T18:37:00.573545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:40:04.775503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":45,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.574383+0000","last_change":"2023-06-23T18:35:48.920009+0000","last_active":"2023-06-23T18:37:00.574383+0000","last_peered":"2023-06-23T18:37:00.574383+0000","last_clean":"2023-06-23T18:37:00.574383+0000","last_became_active":"2023-06-23T18:35:48.919686+0000","last_became_peered":"2023-06-23T18:35:48.919686+0000","last_unstale":"2023-06-23T18:37:00.574383+0000","last_undegraded":"2023-06-23T18:37:00.574383+0000","last_fullsized":"2023-06-23T18:37:00.574383+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:06:33.573819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555317+0000","last_change":"2023-06-23T18:36:58.571860+0000","last_active":"2023-06-23T18:36:59.555317+0000","last_peered":"2023-06-23T18:36:59.555317+0000","last_clean":"2023-06-23T18:36:59.555317+0000","last_became_active":"2023-06-23T18:36:58.571292+0000","last_became_peered":"2023-06-23T18:36:58.571292+0000","last_unstale":"2023-06-23T18:36:59.555317+0000","last_undegraded":"2023-06-23T18:36:59.555317+0000","last_fullsized":"2023-06-23T18:36:59.555317+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:27:02.694068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.553361+0000","last_change":"2023-06-23T18:36:58.162952+0000","last_active":"2023-06-23T18:36:59.553361+0000","last_peered":"2023-06-23T18:36:59.553361+0000","last_clean":"2023-06-23T18:36:59.553361+0000","last_became_active":"2023-06-23T18:36:58.162672+0000","last_became_peered":"2023-06-23T18:36:58.162672+0000","last_unstale":"2023-06-23T18:36:59.553361+0000","last_undegraded":"2023-06-23T18:36:59.553361+0000","last_fullsized":"2023-06-23T18:36:59.553361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:22:01.197428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.553915+0000","last_change":"2023-06-23T18:35:47.962164+0000","last_active":"2023-06-23T18:36:59.553915+0000","last_peered":"2023-06-23T18:36:59.553915+0000","last_clean":"2023-06-23T18:36:59.553915+0000","last_became_active":"2023-06-23T18:35:47.961790+0000","last_became_peered":"2023-06-23T18:35:47.961790+0000","last_unstale":"2023-06-23T18:36:59.553915+0000","last_undegraded":"2023-06-23T18:36:59.553915+0000","last_fullsized":"2023-06-23T18:36:59.553915+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:15:59.947489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.328261+0000","last_change":"2023-06-23T18:36:58.563574+0000","last_active":"2023-06-23T18:37:00.328261+0000","last_peered":"2023-06-23T18:37:00.328261+0000","last_clean":"2023-06-23T18:37:00.328261+0000","last_became_active":"2023-06-23T18:36:58.563302+0000","last_became_peered":"2023-06-23T18:36:58.563302+0000","last_unstale":"2023-06-23T18:37:00.328261+0000","last_undegraded":"2023-06-23T18:37:00.328261+0000","last_fullsized":"2023-06-23T18:37:00.328261+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:51:01.762211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555978+0000","last_change":"2023-06-23T18:36:58.167536+0000","last_active":"2023-06-23T18:36:59.555978+0000","last_peered":"2023-06-23T18:36:59.555978+0000","last_clean":"2023-06-23T18:36:59.555978+0000","last_became_active":"2023-06-23T18:36:58.167287+0000","last_became_peered":"2023-06-23T18:36:58.167287+0000","last_unstale":"2023-06-23T18:36:59.555978+0000","last_undegraded":"2023-06-23T18:36:59.555978+0000","last_fullsized":"2023-06-23T18:36:59.555978+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:38:21.270779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.328227+0000","last_change":"2023-06-23T18:35:47.965867+0000","last_active":"2023-06-23T18:37:00.328227+0000","last_peered":"2023-06-23T18:37:00.328227+0000","last_clean":"2023-06-23T18:37:00.328227+0000","last_became_active":"2023-06-23T18:35:47.965580+0000","last_became_peered":"2023-06-23T18:35:47.965580+0000","last_unstale":"2023-06-23T18:37:00.328227+0000","last_undegraded":"2023-06-23T18:37:00.328227+0000","last_fullsized":"2023-06-23T18:37:00.328227+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:43:51.259773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":193,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555097+0000","last_change":"2023-06-23T18:35:47.968534+0000","last_active":"2023-06-23T18:36:59.555097+0000","last_peered":"2023-06-23T18:36:59.555097+0000","last_clean":"2023-06-23T18:36:59.555097+0000","last_became_active":"2023-06-23T18:35:47.967508+0000","last_became_peered":"2023-06-23T18:35:47.967508+0000","last_unstale":"2023-06-23T18:36:59.555097+0000","last_undegraded":"2023-06-23T18:36:59.555097+0000","last_fullsized":"2023-06-23T18:36:59.555097+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:34:28.376470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555305+0000","last_change":"2023-06-23T18:36:58.571959+0000","last_active":"2023-06-23T18:36:59.555305+0000","last_peered":"2023-06-23T18:36:59.555305+0000","last_clean":"2023-06-23T18:36:59.555305+0000","last_became_active":"2023-06-23T18:36:58.571566+0000","last_became_peered":"2023-06-23T18:36:58.571566+0000","last_unstale":"2023-06-23T18:36:59.555305+0000","last_undegraded":"2023-06-23T18:36:59.555305+0000","last_fullsized":"2023-06-23T18:36:59.555305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:40:19.253293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.408686+0000","last_change":"2023-06-23T18:36:58.160650+0000","last_active":"2023-06-23T18:37:00.408686+0000","last_peered":"2023-06-23T18:37:00.408686+0000","last_clean":"2023-06-23T18:37:00.408686+0000","last_became_active":"2023-06-23T18:36:58.159651+0000","last_became_peered":"2023-06-23T18:36:58.159651+0000","last_unstale":"2023-06-23T18:37:00.408686+0000","last_undegraded":"2023-06-23T18:37:00.408686+0000","last_fullsized":"2023-06-23T18:37:00.408686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:56:22.749779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554348+0000","last_change":"2023-06-23T18:35:48.736407+0000","last_active":"2023-06-23T18:36:59.554348+0000","last_peered":"2023-06-23T18:36:59.554348+0000","last_clean":"2023-06-23T18:36:59.554348+0000","last_became_active":"2023-06-23T18:35:48.735621+0000","last_became_peered":"2023-06-23T18:35:48.735621+0000","last_unstale":"2023-06-23T18:36:59.554348+0000","last_undegraded":"2023-06-23T18:36:59.554348+0000","last_fullsized":"2023-06-23T18:36:59.554348+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:49:13.953312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555848+0000","last_change":"2023-06-23T18:36:58.567077+0000","last_active":"2023-06-23T18:36:59.555848+0000","last_peered":"2023-06-23T18:36:59.555848+0000","last_clean":"2023-06-23T18:36:59.555848+0000","last_became_active":"2023-06-23T18:36:58.562956+0000","last_became_peered":"2023-06-23T18:36:58.562956+0000","last_unstale":"2023-06-23T18:36:59.555848+0000","last_undegraded":"2023-06-23T18:36:59.555848+0000","last_fullsized":"2023-06-23T18:36:59.555848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:25:38.456175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.555154+0000","last_change":"2023-06-23T18:36:58.168257+0000","last_active":"2023-06-23T18:36:59.555154+0000","last_peered":"2023-06-23T18:36:59.555154+0000","last_clean":"2023-06-23T18:36:59.555154+0000","last_became_active":"2023-06-23T18:36:58.167946+0000","last_became_peered":"2023-06-23T18:36:58.167946+0000","last_unstale":"2023-06-23T18:36:59.555154+0000","last_undegraded":"2023-06-23T18:36:59.555154+0000","last_fullsized":"2023-06-23T18:36:59.555154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:47:13.491090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554934+0000","last_change":"2023-06-23T18:35:48.736449+0000","last_active":"2023-06-23T18:36:59.554934+0000","last_peered":"2023-06-23T18:36:59.554934+0000","last_clean":"2023-06-23T18:36:59.554934+0000","last_became_active":"2023-06-23T18:35:48.735664+0000","last_became_peered":"2023-06-23T18:35:48.735664+0000","last_unstale":"2023-06-23T18:36:59.554934+0000","last_undegraded":"2023-06-23T18:36:59.554934+0000","last_fullsized":"2023-06-23T18:36:59.554934+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:57:43.945296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.553811+0000","last_change":"2023-06-23T18:36:58.561017+0000","last_active":"2023-06-23T18:36:59.553811+0000","last_peered":"2023-06-23T18:36:59.553811+0000","last_clean":"2023-06-23T18:36:59.553811+0000","last_became_active":"2023-06-23T18:36:58.560754+0000","last_became_peered":"2023-06-23T18:36:58.560754+0000","last_unstale":"2023-06-23T18:36:59.553811+0000","last_undegraded":"2023-06-23T18:36:59.553811+0000","last_fullsized":"2023-06-23T18:36:59.553811+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:36:32.673996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573498+0000","last_change":"2023-06-23T18:36:58.163646+0000","last_active":"2023-06-23T18:37:00.573498+0000","last_peered":"2023-06-23T18:37:00.573498+0000","last_clean":"2023-06-23T18:37:00.573498+0000","last_became_active":"2023-06-23T18:36:58.163361+0000","last_became_peered":"2023-06-23T18:36:58.163361+0000","last_unstale":"2023-06-23T18:37:00.573498+0000","last_undegraded":"2023-06-23T18:37:00.573498+0000","last_fullsized":"2023-06-23T18:37:00.573498+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:38:27.270329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":45,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:36:59.554770+0000","last_change":"2023-06-23T18:35:48.735713+0000","last_active":"2023-06-23T18:36:59.554770+0000","last_peered":"2023-06-23T18:36:59.554770+0000","last_clean":"2023-06-23T18:36:59.554770+0000","last_became_active":"2023-06-23T18:35:48.735236+0000","last_became_peered":"2023-06-23T18:35:48.735236+0000","last_unstale":"2023-06-23T18:36:59.554770+0000","last_undegraded":"2023-06-23T18:36:59.554770+0000","last_fullsized":"2023-06-23T18:36:59.554770+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:52:07.019238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.328118+0000","last_change":"2023-06-23T18:36:58.565919+0000","last_active":"2023-06-23T18:37:00.328118+0000","last_peered":"2023-06-23T18:37:00.328118+0000","last_clean":"2023-06-23T18:37:00.328118+0000","last_became_active":"2023-06-23T18:36:58.565675+0000","last_became_peered":"2023-06-23T18:36:58.565675+0000","last_unstale":"2023-06-23T18:37:00.328118+0000","last_undegraded":"2023-06-23T18:37:00.328118+0000","last_fullsized":"2023-06-23T18:37:00.328118+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:08:13.663280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.574027+0000","last_change":"2023-06-23T18:36:58.165895+0000","last_active":"2023-06-23T18:37:00.574027+0000","last_peered":"2023-06-23T18:37:00.574027+0000","last_clean":"2023-06-23T18:37:00.574027+0000","last_became_active":"2023-06-23T18:36:58.165576+0000","last_became_peered":"2023-06-23T18:36:58.165576+0000","last_unstale":"2023-06-23T18:37:00.574027+0000","last_undegraded":"2023-06-23T18:37:00.574027+0000","last_fullsized":"2023-06-23T18:37:00.574027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:19:39.994601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573898+0000","last_change":"2023-06-23T18:35:48.735631+0000","last_active":"2023-06-23T18:37:00.573898+0000","last_peered":"2023-06-23T18:37:00.573898+0000","last_clean":"2023-06-23T18:37:00.573898+0000","last_became_active":"2023-06-23T18:35:48.735255+0000","last_became_peered":"2023-06-23T18:35:48.735255+0000","last_unstale":"2023-06-23T18:37:00.573898+0000","last_undegraded":"2023-06-23T18:37:00.573898+0000","last_fullsized":"2023-06-23T18:37:00.573898+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:10:09.040742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573476+0000","last_change":"2023-06-23T18:36:58.562201+0000","last_active":"2023-06-23T18:37:00.573476+0000","last_peered":"2023-06-23T18:37:00.573476+0000","last_clean":"2023-06-23T18:37:00.573476+0000","last_became_active":"2023-06-23T18:36:58.561852+0000","last_became_peered":"2023-06-23T18:36:58.561852+0000","last_unstale":"2023-06-23T18:37:00.573476+0000","last_undegraded":"2023-06-23T18:37:00.573476+0000","last_fullsized":"2023-06-23T18:37:00.573476+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:58.143797+0000","last_clean_scrub_stamp":"2023-06-23T18:36:58.143797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:17:36.608128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.573340+0000","last_change":"2023-06-23T18:36:58.163157+0000","last_active":"2023-06-23T18:37:00.573340+0000","last_peered":"2023-06-23T18:37:00.573340+0000","last_clean":"2023-06-23T18:37:00.573340+0000","last_became_active":"2023-06-23T18:36:58.162834+0000","last_became_peered":"2023-06-23T18:36:58.162834+0000","last_unstale":"2023-06-23T18:37:00.573340+0000","last_undegraded":"2023-06-23T18:37:00.573340+0000","last_fullsized":"2023-06-23T18:37:00.573340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:36:57.137463+0000","last_clean_scrub_stamp":"2023-06-23T18:36:57.137463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:37:30.505833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-23T18:37:00.574052+0000","last_change":"2023-06-23T18:35:47.970359+0000","last_active":"2023-06-23T18:37:00.574052+0000","last_peered":"2023-06-23T18:37:00.574052+0000","last_clean":"2023-06-23T18:37:00.574052+0000","last_became_active":"2023-06-23T18:35:47.970118+0000","last_became_peered":"2023-06-23T18:35:47.970118+0000","last_unstale":"2023-06-23T18:37:00.574052+0000","last_undegraded":"2023-06-23T18:37:00.574052+0000","last_fullsized":"2023-06-23T18:37:00.574052+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:08:13.778628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":991232,"data_stored":750858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":11.071,"5min":11.071,"15min":11.071},"last":0.51100000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":11.118,"5min":11.118,"15min":11.118},"last":0.41299999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":11.103999999999999,"5min":11.103999999999999,"15min":11.103999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.499},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.59399999999999997}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.45100000000000001}]},{"osd":5,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.437}]},{"osd":6,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.64400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":160799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41199999999999998}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.432}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.65800000000000003}]},{"osd":5,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.64000000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":970752,"data_stored":744022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.8680000000000003,"5min":7.8680000000000003,"15min":7.8680000000000003},"last":0.307}]},{"osd":2,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.6429999999999998,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":9.4209999999999994,"5min":9.4209999999999994,"15min":9.4209999999999994},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":5.6790000000000003},"last":0.628},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":7.9039999999999999,"5min":7.9039999999999999,"15min":7.9039999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":11.385999999999999,"5min":11.385999999999999,"15min":11.385999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":7.9649999999999999,"5min":7.9649999999999999,"15min":7.9649999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":11.259,"5min":11.259,"15min":11.259},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":7.9909999999999997,"5min":7.9909999999999997,"15min":7.9909999999999997},"last":0.505},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":11.346,"5min":11.346,"15min":11.346},"last":0.60799999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":7.9359999999999999,"5min":7.9359999999999999,"15min":7.9359999999999999},"last":0.64400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":158890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":16.004999999999999,"5min":16.004999999999999,"15min":16.004999999999999},"last":0.19700000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":16.073,"5min":16.073,"15min":16.073},"last":0.25700000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":8.2460000000000004,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":8.2129999999999992,"5min":8.2129999999999992,"15min":8.2129999999999992},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.23799999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":16.036999999999999,"5min":16.036999999999999,"15min":16.036999999999999},"last":0.32500000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":8.0600000000000005,"5min":8.0600000000000005,"15min":8.0600000000000005},"last":0.437},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":8.2739999999999991,"5min":8.2739999999999991,"15min":8.2739999999999991},"last":0.46999999999999997}]},{"osd":5,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":8.3320000000000007,"5min":8.3320000000000007,"15min":8.3320000000000007},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":8.0299999999999994,"5min":8.0299999999999994,"15min":8.0299999999999994},"last":0.56200000000000006}]},{"osd":6,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.3030000000000008,"5min":8.3030000000000008,"15min":8.3030000000000008},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":8.3800000000000008,"5min":8.3800000000000008,"15min":8.3800000000000008},"last":0.55300000000000005}]},{"osd":7,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":8.3580000000000005,"5min":8.3580000000000005,"15min":8.3580000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":8.0839999999999996,"5min":8.0839999999999996,"15min":8.0839999999999996},"last":0.47899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":158926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":5.0899999999999999,"5min":5.0899999999999999,"15min":5.0899999999999999},"last":0.438},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.42099999999999999}]},{"osd":4,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.50600000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.65200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":161815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":4.359,"5min":4.359,"15min":4.359},"last":0.34200000000000003}]},{"osd":1,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.35599999999999998}]},{"osd":2,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.54500000000000004}]},{"osd":6,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":380928,"data_stored":154168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.36799999999999999}]},{"osd":1,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.45900000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52200000000000002}]},{"osd":3,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.505}]},{"osd":5,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.35299999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.44800000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.434},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":393216,"data_stored":160329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.51200000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.44},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.38300000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.624},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:37:03.025 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-06-23T18:37:03.026 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-06-23T18:37:03.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.179+0000 7fc8a1b0f700 1 Processor -- start 2023-06-23T18:37:03.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.180+0000 7fc8a1b0f700 1 -- start start 2023-06-23T18:37:03.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.180+0000 7fc8a1b0f700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc89c15c710 0x7fc89c0b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:03.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.180+0000 7fc8a1b0f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc89c0aed90 0x7fc89c0b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:03.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.180+0000 7fc8a1b0f700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c15d620 con 0x7fc89c0cd410 2023-06-23T18:37:03.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.180+0000 7fc8a1b0f700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c0ca870 con 0x7fc89c15c710 2023-06-23T18:37:03.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.180+0000 7fc8a1b0f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0724a0 con 0x7fc89c0aed90 2023-06-23T18:37:03.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.181+0000 7fc89affd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc89c0aed90 0x7fc89c0b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:03.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.181+0000 7fc89b7fe700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc89c15c710 0x7fc89c0b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:03.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.181+0000 7fc89affd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc89c0aed90 0x7fc89c0b1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:34402/0 (socket says 172.21.15.79:34402) 2023-06-23T18:37:03.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.181+0000 7fc89affd700 1 -- 172.21.15.79:0/4099090469 learned_addr learned my addr 172.21.15.79:0/4099090469 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:03.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.181+0000 7fc89bfff700 1 --1- 172.21.15.79:0/4099090469 >> v1:172.21.15.155:6789/0 conn(0x7fc89c0cd410 0x7fc89c06eeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:52946/0 (socket says 172.21.15.79:52946) 2023-06-23T18:37:03.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.181+0000 7fc89b7fe700 1 -- 172.21.15.79:0/4099090469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc89c0aed90 msgr2=0x7fc89c0b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:03.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.181+0000 7fc89b7fe700 1 --2- 172.21.15.79:0/4099090469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc89c0aed90 0x7fc89c0b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.181+0000 7fc89b7fe700 1 -- 172.21.15.79:0/4099090469 >> v1:172.21.15.155:6789/0 conn(0x7fc89c0cd410 legacy=0x7fc89c06eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:03.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.181+0000 7fc89b7fe700 1 -- 172.21.15.79:0/4099090469 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c0cd9d0 con 0x7fc89c15c710 2023-06-23T18:37:03.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.181+0000 7fc89a7fc700 1 -- 172.21.15.79:0/4099090469 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2832947639 0 0) 0x7fc89c15d620 con 0x7fc89c0cd410 2023-06-23T18:37:03.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.181+0000 7fc89affd700 1 --2- 172.21.15.79:0/4099090469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc89c0aed90 0x7fc89c0b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:03.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.181+0000 7fc89b7fe700 1 --2- 172.21.15.79:0/4099090469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc89c15c710 0x7fc89c0b12c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc890009840 tx=0x7fc890009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=618af2ef4c7c3262 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:03.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.182+0000 7fc89a7fc700 1 -- 172.21.15.79:0/4099090469 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890011830 con 0x7fc89c15c710 2023-06-23T18:37:03.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.182+0000 7fc89a7fc700 1 -- 172.21.15.79:0/4099090469 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc890011990 con 0x7fc89c15c710 2023-06-23T18:37:03.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.182+0000 7fc89a7fc700 1 -- 172.21.15.79:0/4099090469 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890011d20 con 0x7fc89c15c710 2023-06-23T18:37:03.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.182+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/4099090469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc89c15c710 msgr2=0x7fc89c0b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:03.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.182+0000 7fc8a1b0f700 1 --2- 172.21.15.79:0/4099090469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc89c15c710 0x7fc89c0b12c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc890009840 tx=0x7fc890009b50 comp rx=0 tx=0).stop 2023-06-23T18:37:03.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.182+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/4099090469 shutdown_connections 2023-06-23T18:37:03.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.182+0000 7fc8a1b0f700 1 --2- 172.21.15.79:0/4099090469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc89c15c710 0x7fc89c0b12c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.182+0000 7fc8a1b0f700 1 --2- 172.21.15.79:0/4099090469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc89c0aed90 0x7fc89c0b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.182+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/4099090469 >> 172.21.15.79:0/4099090469 conn(0x7fc89c06e200 msgr2=0x7fc89c0cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:03.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.182+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/4099090469 shutdown_connections 2023-06-23T18:37:03.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.182+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/4099090469 wait complete. 2023-06-23T18:37:03.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.183+0000 7fc8a1b0f700 1 Processor -- start 2023-06-23T18:37:03.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.183+0000 7fc8a1b0f700 1 -- start start 2023-06-23T18:37:03.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.184+0000 7fc8a1b0f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc89c0aed90 0x7fc89c1ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:03.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.184+0000 7fc8a1b0f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc89c0cd410 0x7fc89c156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:03.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.184+0000 7fc8a1b0f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc89c15c710 0x7fc89c1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:03.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.184+0000 7fc8a1b0f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c157bc0 con 0x7fc89c0aed90 2023-06-23T18:37:03.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.184+0000 7fc8a1b0f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c155230 con 0x7fc89c15c710 2023-06-23T18:37:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.184+0000 7fc8a1b0f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c155510 con 0x7fc89c0cd410 2023-06-23T18:37:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.184+0000 7fc89b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc89c0aed90 0x7fc89c1ecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.184+0000 7fc89b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc89c0aed90 0x7fc89c1ecf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41638/0 (socket says 172.21.15.79:41638) 2023-06-23T18:37:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.184+0000 7fc89b7fe700 1 -- 172.21.15.79:0/528490585 learned_addr learned my addr 172.21.15.79:0/528490585 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.184+0000 7fc89bfff700 1 --2- 172.21.15.79:0/528490585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc89c15c710 0x7fc89c1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.184+0000 7fc89affd700 1 --2- 172.21.15.79:0/528490585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc89c0cd410 0x7fc89c156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.185+0000 7fc89b7fe700 1 -- 172.21.15.79:0/528490585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc89c0cd410 msgr2=0x7fc89c156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.185+0000 7fc89b7fe700 1 --2- 172.21.15.79:0/528490585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc89c0cd410 0x7fc89c156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.185+0000 7fc89b7fe700 1 -- 172.21.15.79:0/528490585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc89c15c710 msgr2=0x7fc89c1572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.185+0000 7fc89b7fe700 1 --2- 172.21.15.79:0/528490585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc89c15c710 0x7fc89c1572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.185+0000 7fc89b7fe700 1 -- 172.21.15.79:0/528490585 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c00b7f0 con 0x7fc89c0aed90 2023-06-23T18:37:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.185+0000 7fc89b7fe700 1 --2- 172.21.15.79:0/528490585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc89c0aed90 0x7fc89c1ecf10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc890009b90 tx=0x7fc890004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.185+0000 7fc883fff700 1 -- 172.21.15.79:0/528490585 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8900036a0 con 0x7fc89c0aed90 2023-06-23T18:37:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.185+0000 7fc883fff700 1 -- 172.21.15.79:0/528490585 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89001e070 con 0x7fc89c0aed90 2023-06-23T18:37:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.185+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/528490585 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8900094a0 con 0x7fc89c0aed90 2023-06-23T18:37:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.185+0000 7fc883fff700 1 -- 172.21.15.79:0/528490585 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890003a60 con 0x7fc89c0aed90 2023-06-23T18:37:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.185+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/528490585 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c155ad0 con 0x7fc89c0aed90 2023-06-23T18:37:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.186+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/528490585 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c06bf30 con 0x7fc89c0aed90 2023-06-23T18:37:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.186+0000 7fc883fff700 1 -- 172.21.15.79:0/528490585 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc890003bc0 con 0x7fc89c0aed90 2023-06-23T18:37:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.187+0000 7fc883fff700 1 --2- 172.21.15.79:0/528490585 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc88806ace0 0x7fc88806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.187+0000 7fc883fff700 1 -- 172.21.15.79:0/528490585 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc89008a310 con 0x7fc89c0aed90 2023-06-23T18:37:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.187+0000 7fc89affd700 1 --2- 172.21.15.79:0/528490585 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc88806ace0 0x7fc88806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.187+0000 7fc89affd700 1 --2- 172.21.15.79:0/528490585 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc88806ace0 0x7fc88806d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc89c1586c0 tx=0x7fc88c00b4d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:03.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.191+0000 7fc883fff700 1 -- 172.21.15.79:0/528490585 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc890055450 con 0x7fc89c0aed90 2023-06-23T18:37:03.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.453+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/528490585 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc89c0cbc20 con 0x7fc89c0aed90 2023-06-23T18:37:03.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.457+0000 7fc883fff700 1 -- 172.21.15.79:0/528490585 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc890058a70 con 0x7fc89c0aed90 2023-06-23T18:37:03.462 INFO:teuthology.orchestra.run.smithi079.stdout:[client.0] 2023-06-23T18:37:03.462 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDV5ZVkbyA+LRAA4TQMvihkdeAgh/w8764lVA== 2023-06-23T18:37:03.462 INFO:teuthology.orchestra.run.smithi079.stdout: caps mds = "allow" 2023-06-23T18:37:03.462 INFO:teuthology.orchestra.run.smithi079.stdout: caps mon = "allow r" 2023-06-23T18:37:03.462 INFO:teuthology.orchestra.run.smithi079.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-06-23T18:37:03.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.459+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/528490585 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc88806ace0 msgr2=0x7fc88806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:03.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.459+0000 7fc8a1b0f700 1 --2- 172.21.15.79:0/528490585 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc88806ace0 0x7fc88806d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc89c1586c0 tx=0x7fc88c00b4d0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.459+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/528490585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc89c0aed90 msgr2=0x7fc89c1ecf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:03.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.459+0000 7fc8a1b0f700 1 --2- 172.21.15.79:0/528490585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc89c0aed90 0x7fc89c1ecf10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc890009b90 tx=0x7fc890004750 comp rx=0 tx=0).stop 2023-06-23T18:37:03.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.459+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/528490585 shutdown_connections 2023-06-23T18:37:03.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.459+0000 7fc8a1b0f700 1 --2- 172.21.15.79:0/528490585 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fc88806ace0 0x7fc88806d190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.459+0000 7fc8a1b0f700 1 --2- 172.21.15.79:0/528490585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc89c15c710 0x7fc89c1572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.459+0000 7fc8a1b0f700 1 --2- 172.21.15.79:0/528490585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc89c0aed90 0x7fc89c1ecf10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.459+0000 7fc8a1b0f700 1 --2- 172.21.15.79:0/528490585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc89c0cd410 0x7fc89c156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.459+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/528490585 >> 172.21.15.79:0/528490585 conn(0x7fc89c06e200 msgr2=0x7fc89c0af200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:03.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.459+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/528490585 shutdown_connections 2023-06-23T18:37:03.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.459+0000 7fc8a1b0f700 1 -- 172.21.15.79:0/528490585 wait complete. 2023-06-23T18:37:03.466 INFO:teuthology.orchestra.run.smithi079.stderr:updated caps for client.0 2023-06-23T18:37:03.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:37:03.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.592+0000 7fcfc7a74700 1 Processor -- start 2023-06-23T18:37:03.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.592+0000 7fcfc7a74700 1 -- start start 2023-06-23T18:37:03.599 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc7a74700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfc0159170 0x7fcfc00cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:03.599 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc7a74700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcfc00ce7f0 0x7fcfc00cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:03.599 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc7a74700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc015cd50 con 0x7fcfc00cd900 2023-06-23T18:37:03.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc7a74700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc00cdd70 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc7a74700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc015cfa0 con 0x7fcfc0159170 2023-06-23T18:37:03.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc6011700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcfc00ce7f0 0x7fcfc00cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:03.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc500f700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fcfc00cd900 0x7fcfc00ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44688/0 (socket says 172.21.15.79:44688) 2023-06-23T18:37:03.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc500f700 1 -- 172.21.15.79:0/2282167237 learned_addr learned my addr 172.21.15.79:0/2282167237 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:03.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc5810700 1 --2- 172.21.15.79:0/2282167237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfc0159170 0x7fcfc00cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:03.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc6011700 1 -- 172.21.15.79:0/2282167237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfc0159170 msgr2=0x7fcfc00cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:03.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc6011700 1 --2- 172.21.15.79:0/2282167237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfc0159170 0x7fcfc00cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc6011700 1 -- 172.21.15.79:0/2282167237 >> v1:172.21.15.79:6789/0 conn(0x7fcfc00cd900 legacy=0x7fcfc00ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:03.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc6011700 1 -- 172.21.15.79:0/2282167237 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc015d280 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc5810700 1 --2- 172.21.15.79:0/2282167237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfc0159170 0x7fcfc00cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:03.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.593+0000 7fcfc6011700 1 --2- 172.21.15.79:0/2282167237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfc00ce7f0 0x7fcfc00cc1d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009d90 tx=0x7fcfb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4afe8acdeece19a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:03.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.594+0000 7fcfc480e700 1 -- 172.21.15.79:0/2282167237 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800e070 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.594+0000 7fcfc480e700 1 -- 172.21.15.79:0/2282167237 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb800bbd0 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.594+0000 7fcfc480e700 1 -- 172.21.15.79:0/2282167237 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8012430 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.594+0000 7fcfc7a74700 1 -- 172.21.15.79:0/2282167237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfc00ce7f0 msgr2=0x7fcfc00cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:03.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.594+0000 7fcfc7a74700 1 --2- 172.21.15.79:0/2282167237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfc00ce7f0 0x7fcfc00cc1d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009d90 tx=0x7fcfb800b330 comp rx=0 tx=0).stop 2023-06-23T18:37:03.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.594+0000 7fcfc7a74700 1 -- 172.21.15.79:0/2282167237 shutdown_connections 2023-06-23T18:37:03.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.594+0000 7fcfc7a74700 1 --2- 172.21.15.79:0/2282167237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfc00ce7f0 0x7fcfc00cc1d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.594+0000 7fcfc7a74700 1 --2- 172.21.15.79:0/2282167237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfc0159170 0x7fcfc00cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.594+0000 7fcfc7a74700 1 -- 172.21.15.79:0/2282167237 >> 172.21.15.79:0/2282167237 conn(0x7fcfc006d660 msgr2=0x7fcfc006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:03.606 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.594+0000 7fcfc7a74700 1 -- 172.21.15.79:0/2282167237 shutdown_connections 2023-06-23T18:37:03.606 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.594+0000 7fcfc7a74700 1 -- 172.21.15.79:0/2282167237 wait complete. 2023-06-23T18:37:03.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.595+0000 7fcfc7a74700 1 Processor -- start 2023-06-23T18:37:03.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.595+0000 7fcfc7a74700 1 -- start start 2023-06-23T18:37:03.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.596+0000 7fcfc7a74700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfc00cd900 0x7fcfc00c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:03.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.596+0000 7fcfc7a74700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfc00ce7f0 0x7fcfc0148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:03.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.596+0000 7fcfc7a74700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfc0159170 0x7fcfc00b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:03.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.596+0000 7fcfc7a74700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc00c6e00 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.596+0000 7fcfc7a74700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc00c3030 con 0x7fcfc00cd900 2023-06-23T18:37:03.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.596+0000 7fcfc7a74700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc00c8ef0 con 0x7fcfc0159170 2023-06-23T18:37:03.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.596+0000 7fcfc500f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfc00ce7f0 0x7fcfc0148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:03.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.596+0000 7fcfc500f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfc00ce7f0 0x7fcfc0148c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41654/0 (socket says 172.21.15.79:41654) 2023-06-23T18:37:03.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.596+0000 7fcfc500f700 1 -- 172.21.15.79:0/845920581 learned_addr learned my addr 172.21.15.79:0/845920581 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:03.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.596+0000 7fcfc5810700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfc00cd900 0x7fcfc00c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:03.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.596+0000 7fcfc6011700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfc0159170 0x7fcfc00b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:03.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.597+0000 7fcfc500f700 1 -- 172.21.15.79:0/845920581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfc0159170 msgr2=0x7fcfc00b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:03.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.597+0000 7fcfc500f700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfc0159170 0x7fcfc00b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.597+0000 7fcfc500f700 1 -- 172.21.15.79:0/845920581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfc00cd900 msgr2=0x7fcfc00c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:03.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.597+0000 7fcfc500f700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfc00cd900 0x7fcfc00c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.597+0000 7fcfc500f700 1 -- 172.21.15.79:0/845920581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb8009a70 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.597+0000 7fcfc6011700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfc0159170 0x7fcfc00b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.597+0000 7fcfc500f700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfc00ce7f0 0x7fcfc0148c60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c520 tx=0x7fcfb000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.597+0000 7fcfb67fc700 1 -- 172.21.15.79:0/845920581 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0005d70 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.597+0000 7fcfb67fc700 1 -- 172.21.15.79:0/845920581 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb00070d0 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.597+0000 7fcfb67fc700 1 -- 172.21.15.79:0/845920581 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0007440 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.597+0000 7fcfc7a74700 1 -- 172.21.15.79:0/845920581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc00c91d0 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.597+0000 7fcfc7a74700 1 -- 172.21.15.79:0/845920581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc00c9790 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.598+0000 7fcfc7a74700 1 -- 172.21.15.79:0/845920581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc015a860 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.598+0000 7fcfb67fc700 1 -- 172.21.15.79:0/845920581 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcfb00075a0 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.599+0000 7fcfb67fc700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcfa406ace0 0x7fcfa406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:03.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.599+0000 7fcfb67fc700 1 -- 172.21.15.79:0/845920581 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcfb0088740 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.599+0000 7fcfc5810700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcfa406ace0 0x7fcfa406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:03.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.599+0000 7fcfc5810700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcfa406ace0 0x7fcfa406d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc006fd0 tx=0x7fcfbc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:03.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.603+0000 7fcfb67fc700 1 -- 172.21.15.79:0/845920581 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcfc015a860 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.831+0000 7fcfc7a74700 1 -- 172.21.15.79:0/845920581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfc015a860 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.832+0000 7fcfb67fc700 1 -- 172.21.15.79:0/845920581 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4677 (secure 0 0 0) 0x7fcfc015a860 con 0x7fcfc00ce7f0 2023-06-23T18:37:03.837 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:03.837 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3101032943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":3101032943},{"type":"v1","addr":"172.21.15.79:6837","nonce":3101032943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2056827644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2056827644},{"type":"v1","addr":"172.21.15.155:6833","nonce":2056827644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1835571378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1835571378},{"type":"v1","addr":"172.21.15.155:6835","nonce":1835571378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:36:58.545517+0000","modified":"2023-06-23T18:37:01.557968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6836/1030786659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1030786659},{"type":"v1","addr":"172.21.15.79:6836","nonce":1030786659}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-23T18:37:03.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.834+0000 7fcfc7a74700 1 -- 172.21.15.79:0/845920581 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcfa406ace0 msgr2=0x7fcfa406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:03.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.834+0000 7fcfc7a74700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcfa406ace0 0x7fcfa406d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc006fd0 tx=0x7fcfbc008040 comp rx=0 tx=0).stop 2023-06-23T18:37:03.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.834+0000 7fcfc7a74700 1 -- 172.21.15.79:0/845920581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfc00ce7f0 msgr2=0x7fcfc0148c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:03.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.834+0000 7fcfc7a74700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfc00ce7f0 0x7fcfc0148c60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c520 tx=0x7fcfb000c830 comp rx=0 tx=0).stop 2023-06-23T18:37:03.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.834+0000 7fcfc7a74700 1 -- 172.21.15.79:0/845920581 shutdown_connections 2023-06-23T18:37:03.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.834+0000 7fcfc7a74700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfc0159170 0x7fcfc00b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.834+0000 7fcfc7a74700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcfa406ace0 0x7fcfa406d190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.834+0000 7fcfc7a74700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfc00cd900 0x7fcfc00c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.834+0000 7fcfc7a74700 1 --2- 172.21.15.79:0/845920581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfc00ce7f0 0x7fcfc0148c60 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:03.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.834+0000 7fcfc7a74700 1 -- 172.21.15.79:0/845920581 >> 172.21.15.79:0/845920581 conn(0x7fcfc006d660 msgr2=0x7fcfc00af730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:03.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.834+0000 7fcfc7a74700 1 -- 172.21.15.79:0/845920581 shutdown_connections 2023-06-23T18:37:03.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:03.834+0000 7fcfc7a74700 1 -- 172.21.15.79:0/845920581 wait complete. 2023-06-23T18:37:03.841 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 16 2023-06-23T18:37:04.252 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-06-23T18:36:58.545517+0000', 'modified': '2023-06-23T18:37:01.557968+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5292}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5292': {'gid': 5292, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.79:6836/1030786659', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6834', 'nonce': 1030786659}, {'type': 'v1', 'addr': '172.21.15.79:6836', 'nonce': 1030786659}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-06-23T18:37:04.253 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-06-23T18:37:04.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-06-23T18:37:04.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.372+0000 7fa3339f5700 1 Processor -- start 2023-06-23T18:37:04.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.372+0000 7fa3339f5700 1 -- start start 2023-06-23T18:37:04.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.373+0000 7fa3339f5700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa32c15a380 0x7fa32c0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.373+0000 7fa3339f5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa32c15a750 0x7fa32c0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.373+0000 7fa3339f5700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32c0b1d40 con 0x7fa32c0ade50 2023-06-23T18:37:04.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.373+0000 7fa3339f5700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32c0b2020 con 0x7fa32c15a380 2023-06-23T18:37:04.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.373+0000 7fa3339f5700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0ad650 con 0x7fa32c15a750 2023-06-23T18:37:04.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.373+0000 7fa331f92700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa32c0ade50 0x7fa32c0ae290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44706/0 (socket says 172.21.15.79:44706) 2023-06-23T18:37:04.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.373+0000 7fa331f92700 1 -- 172.21.15.79:0/3187407607 learned_addr learned my addr 172.21.15.79:0/3187407607 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:04.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa331791700 1 --2- 172.21.15.79:0/3187407607 >> v2:172.21.15.155:3300/0 conn(0x7fa32c15a380 0x7fa32c0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa330f90700 1 --2- 172.21.15.79:0/3187407607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa32c15a750 0x7fa32c0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 -- 172.21.15.79:0/3187407607 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2581400008 0 0) 0x7fa32c0b1d40 con 0x7fa32c0ade50 2023-06-23T18:37:04.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 -- 172.21.15.79:0/3187407607 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3180035b0 con 0x7fa32c0ade50 2023-06-23T18:37:04.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 -- 172.21.15.79:0/3187407607 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 8063697 0 0) 0x7fa3180035b0 con 0x7fa32c0ade50 2023-06-23T18:37:04.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 -- 172.21.15.79:0/3187407607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa32c15a750 msgr2=0x7fa32c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:04.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 --2- 172.21.15.79:0/3187407607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa32c15a750 0x7fa32c0cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 -- 172.21.15.79:0/3187407607 >> v2:172.21.15.155:3300/0 conn(0x7fa32c15a380 msgr2=0x7fa32c0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:04.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 --2- 172.21.15.79:0/3187407607 >> v2:172.21.15.155:3300/0 conn(0x7fa32c15a380 0x7fa32c0ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 -- 172.21.15.79:0/3187407607 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c0ad930 con 0x7fa32c0ade50 2023-06-23T18:37:04.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 -- 172.21.15.79:0/3187407607 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fa3240019f0 con 0x7fa32c0ade50 2023-06-23T18:37:04.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 -- 172.21.15.79:0/3187407607 >> v1:172.21.15.79:6789/0 conn(0x7fa32c0ade50 legacy=0x7fa32c0ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 --2- 172.21.15.79:0/3187407607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa32c15a750 0x7fa318003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 --2- 172.21.15.79:0/3187407607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa318003f80 0x7fa318004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 --2- 172.21.15.79:0/3187407607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa318004d90 0x7fa318005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 -- 172.21.15.79:0/3187407607 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0b2020 con 0x7fa318003f80 2023-06-23T18:37:04.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 -- 172.21.15.79:0/3187407607 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0ad650 con 0x7fa318004d90 2023-06-23T18:37:04.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa323fff700 1 -- 172.21.15.79:0/3187407607 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31800ba10 con 0x7fa32c15a750 2023-06-23T18:37:04.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa330f90700 1 --2- 172.21.15.79:0/3187407607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa318003f80 0x7fa318004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa331791700 1 --2- 172.21.15.79:0/3187407607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa32c15a750 0x7fa318003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa331f92700 1 --2- 172.21.15.79:0/3187407607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa318004d90 0x7fa318005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa330f90700 1 -- 172.21.15.79:0/3187407607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa32c15a750 msgr2=0x7fa318003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:04.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa330f90700 1 --2- 172.21.15.79:0/3187407607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa32c15a750 0x7fa318003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa330f90700 1 -- 172.21.15.79:0/3187407607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa318004d90 msgr2=0x7fa318005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:04.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa330f90700 1 --2- 172.21.15.79:0/3187407607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa318004d90 0x7fa318005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa330f90700 1 -- 172.21.15.79:0/3187407607 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3180035b0 con 0x7fa318003f80 2023-06-23T18:37:04.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.374+0000 7fa330f90700 1 --2- 172.21.15.79:0/3187407607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa318003f80 0x7fa318004850 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa32c06e090 tx=0x7fa318005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60362265837e11d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:04.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.375+0000 7fa323fff700 1 -- 172.21.15.79:0/3187407607 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32800e6c0 con 0x7fa318003f80 2023-06-23T18:37:04.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.375+0000 7fa323fff700 1 -- 172.21.15.79:0/3187407607 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32800e820 con 0x7fa318003f80 2023-06-23T18:37:04.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.375+0000 7fa323fff700 1 -- 172.21.15.79:0/3187407607 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32800eb90 con 0x7fa318003f80 2023-06-23T18:37:04.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.375+0000 7fa3339f5700 1 -- 172.21.15.79:0/3187407607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa318003f80 msgr2=0x7fa318004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.375+0000 7fa3339f5700 1 --2- 172.21.15.79:0/3187407607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa318003f80 0x7fa318004850 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa32c06e090 tx=0x7fa318005880 comp rx=0 tx=0).stop 2023-06-23T18:37:04.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.375+0000 7fa331f92700 1 -- 172.21.15.79:0/3187407607 reap_dead start 2023-06-23T18:37:04.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.375+0000 7fa3339f5700 1 -- 172.21.15.79:0/3187407607 shutdown_connections 2023-06-23T18:37:04.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.375+0000 7fa3339f5700 1 -- 172.21.15.79:0/3187407607 >> 172.21.15.79:0/3187407607 conn(0x7fa32c089fc0 msgr2=0x7fa32c099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:04.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.375+0000 7fa3339f5700 1 -- 172.21.15.79:0/3187407607 shutdown_connections 2023-06-23T18:37:04.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.375+0000 7fa3339f5700 1 -- 172.21.15.79:0/3187407607 wait complete. 2023-06-23T18:37:04.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.376+0000 7fa3339f5700 1 Processor -- start 2023-06-23T18:37:04.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.376+0000 7fa3339f5700 1 -- start start 2023-06-23T18:37:04.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.377+0000 7fa3339f5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa318003f80 0x7fa32c1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.377+0000 7fa3339f5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa32c0b6200 0x7fa32c1f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.377+0000 7fa3339f5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa32c1e3d90 0x7fa32c1e4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.377+0000 7fa3339f5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c1e4740 con 0x7fa32c0b6200 2023-06-23T18:37:04.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.377+0000 7fa3339f5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0c5710 con 0x7fa318003f80 2023-06-23T18:37:04.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.377+0000 7fa3339f5700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c1f6380 con 0x7fa32c1e3d90 2023-06-23T18:37:04.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.377+0000 7fa330f90700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa32c0b6200 0x7fa32c1f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.377+0000 7fa330f90700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa32c0b6200 0x7fa32c1f16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41674/0 (socket says 172.21.15.79:41674) 2023-06-23T18:37:04.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.377+0000 7fa330f90700 1 -- 172.21.15.79:0/3933574913 learned_addr learned my addr 172.21.15.79:0/3933574913 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:04.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.377+0000 7fa331f92700 1 --2- 172.21.15.79:0/3933574913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa32c1e3d90 0x7fa32c1e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.377+0000 7fa331791700 1 --2- 172.21.15.79:0/3933574913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa318003f80 0x7fa32c1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.378+0000 7fa330f90700 1 -- 172.21.15.79:0/3933574913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa32c1e3d90 msgr2=0x7fa32c1e4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.378+0000 7fa330f90700 1 --2- 172.21.15.79:0/3933574913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa32c1e3d90 0x7fa32c1e4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.378+0000 7fa330f90700 1 -- 172.21.15.79:0/3933574913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa318003f80 msgr2=0x7fa32c1f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.378+0000 7fa330f90700 1 --2- 172.21.15.79:0/3933574913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa318003f80 0x7fa32c1f5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.378+0000 7fa330f90700 1 -- 172.21.15.79:0/3933574913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c0b6780 con 0x7fa32c0b6200 2023-06-23T18:37:04.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.378+0000 7fa330f90700 1 --2- 172.21.15.79:0/3933574913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa32c0b6200 0x7fa32c1f16d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa32800e4b0 tx=0x7fa3280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:04.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.378+0000 7fa321ffb700 1 -- 172.21.15.79:0/3933574913 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32800e6c0 con 0x7fa32c0b6200 2023-06-23T18:37:04.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.378+0000 7fa321ffb700 1 -- 172.21.15.79:0/3933574913 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa328004550 con 0x7fa32c0b6200 2023-06-23T18:37:04.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.378+0000 7fa321ffb700 1 -- 172.21.15.79:0/3933574913 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328006e60 con 0x7fa32c0b6200 2023-06-23T18:37:04.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.378+0000 7fa3339f5700 1 -- 172.21.15.79:0/3933574913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c0c4de0 con 0x7fa32c0b6200 2023-06-23T18:37:04.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.378+0000 7fa3339f5700 1 -- 172.21.15.79:0/3933574913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c0c53a0 con 0x7fa32c0b6200 2023-06-23T18:37:04.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.380+0000 7fa321ffb700 1 -- 172.21.15.79:0/3933574913 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa328003af0 con 0x7fa32c0b6200 2023-06-23T18:37:04.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.380+0000 7fa321ffb700 1 --2- 172.21.15.79:0/3933574913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa31006ace0 0x7fa31006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.380+0000 7fa321ffb700 1 -- 172.21.15.79:0/3933574913 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa32808a6b0 con 0x7fa32c0b6200 2023-06-23T18:37:04.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.380+0000 7fa331791700 1 --2- 172.21.15.79:0/3933574913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa31006ace0 0x7fa31006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.380+0000 7fa331791700 1 --2- 172.21.15.79:0/3933574913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa31006ace0 0x7fa31006d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009bd0 tx=0x7fa31c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:04.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.382+0000 7fa33fb7ab80 1 Processor -- start 2023-06-23T18:37:04.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.382+0000 7fa33fb7ab80 1 -- start start 2023-06-23T18:37:04.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa33fb7ab80 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132aff50 0x5572132b0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa33fb7ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132d7d80 0x5572132d6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa33fb7ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132ad740 0x5572132da440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa33fb7ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572131b4c60 con 0x5572132d7d80 2023-06-23T18:37:04.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa33fb7ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572132079f0 con 0x5572132ad740 2023-06-23T18:37:04.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa33fb7ab80 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x557213200e80 con 0x5572132aff50 2023-06-23T18:37:04.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa330f90700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132d7d80 0x5572132d6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa330f90700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132d7d80 0x5572132d6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41682/0 (socket says 172.21.15.79:41682) 2023-06-23T18:37:04.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa330f90700 1 -- 172.21.15.79:0/1740780199 learned_addr learned my addr 172.21.15.79:0/1740780199 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:04.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa331791700 1 --2- 172.21.15.79:0/1740780199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132aff50 0x5572132b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa331f92700 1 --2- 172.21.15.79:0/1740780199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132ad740 0x5572132da440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa330f90700 1 -- 172.21.15.79:0/1740780199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132aff50 msgr2=0x5572132b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:04.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa330f90700 1 --2- 172.21.15.79:0/1740780199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132aff50 0x5572132b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa330f90700 1 -- 172.21.15.79:0/1740780199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132ad740 msgr2=0x5572132da440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:04.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa330f90700 1 --2- 172.21.15.79:0/1740780199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132ad740 0x5572132da440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa330f90700 1 -- 172.21.15.79:0/1740780199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572130825e0 con 0x5572132d7d80 2023-06-23T18:37:04.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa331f92700 1 --2- 172.21.15.79:0/1740780199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132ad740 0x5572132da440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:04.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa331791700 1 --2- 172.21.15.79:0/1740780199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132aff50 0x5572132b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:04.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa330f90700 1 --2- 172.21.15.79:0/1740780199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132d7d80 0x5572132d6820 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa328088b30 tx=0x7fa328058fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d239204c04ea7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:04.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa30effd700 1 -- 172.21.15.79:0/1740780199 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328088ea0 con 0x5572132d7d80 2023-06-23T18:37:04.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa30effd700 1 -- 172.21.15.79:0/1740780199 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa328003da0 con 0x5572132d7d80 2023-06-23T18:37:04.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.383+0000 7fa30effd700 1 -- 172.21.15.79:0/1740780199 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3280096f0 con 0x5572132d7d80 2023-06-23T18:37:04.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.384+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1740780199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132d7d80 msgr2=0x5572132d6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.384+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1740780199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132d7d80 0x5572132d6820 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa328088b30 tx=0x7fa328058fb0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.384+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1740780199 shutdown_connections 2023-06-23T18:37:04.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.384+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1740780199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132ad740 0x5572132da440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.384+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1740780199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132aff50 0x5572132b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.384+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1740780199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132d7d80 0x5572132d6820 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.384+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1740780199 >> 172.21.15.79:0/1740780199 conn(0x5572132a4fc0 msgr2=0x5572132f51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:04.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.384+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1740780199 shutdown_connections 2023-06-23T18:37:04.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.384+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1740780199 wait complete. 2023-06-23T18:37:04.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.385+0000 7fa33fb7ab80 1 -- start start 2023-06-23T18:37:04.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.385+0000 7fa33fb7ab80 1 Processor -- start 2023-06-23T18:37:04.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa33fb7ab80 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132ad740 0x5572132d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa33fb7ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132aff50 0x5572132f3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa33fb7ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132d7d80 0x5572132ed490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa33fb7ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572131a0a70 con 0x5572132aff50 2023-06-23T18:37:04.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa33fb7ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572131a0770 con 0x5572132d7d80 2023-06-23T18:37:04.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa33fb7ab80 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572130848d0 con 0x5572132ad740 2023-06-23T18:37:04.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa33fb7ab80 10 client.0 _ll_register_callbacks cb 0x5572131a6210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:37:04.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa330f90700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132aff50 0x5572132f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa330f90700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132aff50 0x5572132f3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41688/0 (socket says 172.21.15.79:41688) 2023-06-23T18:37:04.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa330f90700 1 -- 172.21.15.79:0/1866835662 learned_addr learned my addr 172.21.15.79:0/1866835662 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:04.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa331f92700 1 --2- 172.21.15.79:0/1866835662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132d7d80 0x5572132ed490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa331791700 1 --2- 172.21.15.79:0/1866835662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132ad740 0x5572132d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa330f90700 1 -- 172.21.15.79:0/1866835662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132ad740 msgr2=0x5572132d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:04.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa330f90700 1 --2- 172.21.15.79:0/1866835662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132ad740 0x5572132d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa330f90700 1 -- 172.21.15.79:0/1866835662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132d7d80 msgr2=0x5572132ed490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:04.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa330f90700 1 --2- 172.21.15.79:0/1866835662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132d7d80 0x5572132ed490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa330f90700 1 -- 172.21.15.79:0/1866835662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c0c53a0 con 0x5572132aff50 2023-06-23T18:37:04.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa330f90700 1 --2- 172.21.15.79:0/1866835662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132aff50 0x5572132f3830 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa328059560 tx=0x7fa3280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7961161fc4648a00 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:04.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa307fff700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2023-06-23T18:37:04.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa307fff700 1 -- 172.21.15.79:0/1866835662 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328088ea0 con 0x5572132aff50 2023-06-23T18:37:04.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa307fff700 1 -- 172.21.15.79:0/1866835662 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3280096f0 con 0x5572132aff50 2023-06-23T18:37:04.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1866835662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572132a2780 con 0x5572132aff50 2023-06-23T18:37:04.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa307fff700 1 -- 172.21.15.79:0/1866835662 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa328058ac0 con 0x5572132aff50 2023-06-23T18:37:04.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa33fb7ab80 10 client.5487 fetch_fsmap learned FSMap version 16 2023-06-23T18:37:04.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.386+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1866835662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55721302ed90 con 0x5572132aff50 2023-06-23T18:37:04.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.387+0000 7fa307fff700 1 -- 172.21.15.79:0/1866835662 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5572132a2780 con 0x5572132aff50 2023-06-23T18:37:04.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.387+0000 7fa307fff700 1 -- 172.21.15.79:0/1866835662 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa328061490 con 0x5572132aff50 2023-06-23T18:37:04.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.387+0000 7fa33fb7ab80 10 client.5487 fetch_fsmap finished waiting for FSMap version 16 2023-06-23T18:37:04.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.387+0000 7fa33fb7ab80 10 client.5487 resolve_mds: resolved name 'c' to daemon mds.c 2023-06-23T18:37:04.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.387+0000 7fa33fb7ab80 20 client.5487 populate_metadata read hostname 'smithi079' 2023-06-23T18:37:04.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.387+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1866835662 >> [v2:172.21.15.79:6834/1030786659,v1:172.21.15.79:6836/1030786659] conn(0x5572132f8b30 0x5572132ee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.387+0000 7fa33fb7ab80 4 client.5487 mds_command: new command op to 5292 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:37:04.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.387+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1866835662 --> [v2:172.21.15.79:6834/1030786659,v1:172.21.15.79:6836/1030786659] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572131f9e60 con 0x5572132f8b30 2023-06-23T18:37:04.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.387+0000 7fa331791700 1 --2- 172.21.15.79:0/1866835662 >> [v2:172.21.15.79:6834/1030786659,v1:172.21.15.79:6836/1030786659] conn(0x5572132f8b30 0x5572132ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.387+0000 7fa307fff700 10 client.5487 ms_handle_connect on v2:172.21.15.79:6834/1030786659 2023-06-23T18:37:04.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.387+0000 7fa331791700 1 --2- 172.21.15.79:0/1866835662 >> [v2:172.21.15.79:6834/1030786659,v1:172.21.15.79:6836/1030786659] conn(0x5572132f8b30 0x5572132ee710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9827cb8a58ab1d27 server_cookie=f128159db8a9a187 in_seq=0 out_seq=0 2023-06-23T18:37:04.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa307fff700 1 -- 172.21.15.79:0/1866835662 <== mds.0 v2:172.21.15.79:6834/1030786659 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5572131f9e60 con 0x5572132f8b30 2023-06-23T18:37:04.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa307fff700 10 client.5487 handle_command_reply: tid=0 2023-06-23T18:37:04.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa33fb7ab80 1 client.5487 shutdown 2023-06-23T18:37:04.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1866835662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132aff50 msgr2=0x5572132f3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1866835662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132aff50 0x5572132f3830 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa328059560 tx=0x7fa3280046c0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1866835662 shutdown_connections 2023-06-23T18:37:04.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1866835662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132d7d80 0x5572132ed490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1866835662 >> [v2:172.21.15.79:6834/1030786659,v1:172.21.15.79:6836/1030786659] conn(0x5572132f8b30 0x5572132ee710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1866835662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132ad740 0x5572132d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa307fff700 0 client.5487 ms_handle_reset on v2:172.21.15.79:6834/1030786659 2023-06-23T18:37:04.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1866835662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132aff50 0x5572132f3830 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1866835662 >> 172.21.15.79:0/1866835662 conn(0x5572132a4fc0 msgr2=0x5572132a8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:04.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1866835662 shutdown_connections 2023-06-23T18:37:04.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1866835662 wait complete. 2023-06-23T18:37:04.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.388+0000 7fa33fb7ab80 20 client.5487 trim_cache size 0 max 16384 2023-06-23T18:37:04.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.406+0000 7fa33fb7ab80 1 Processor -- start 2023-06-23T18:37:04.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.406+0000 7fa33fb7ab80 1 -- start start 2023-06-23T18:37:04.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.406+0000 7fa33fb7ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132ad740 0x5572132aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.406+0000 7fa33fb7ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132aff50 0x5572132af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.406+0000 7fa33fb7ab80 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132d7d80 0x5572132f1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.406+0000 7fa33fb7ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572132ecff0 con 0x5572132ad740 2023-06-23T18:37:04.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.406+0000 7fa33fb7ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572132f3390 con 0x5572132aff50 2023-06-23T18:37:04.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.406+0000 7fa33fb7ab80 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572132d7750 con 0x5572132d7d80 2023-06-23T18:37:04.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.406+0000 7fa331791700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132ad740 0x5572132aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.407+0000 7fa331791700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132ad740 0x5572132aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41694/0 (socket says 172.21.15.79:41694) 2023-06-23T18:37:04.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.407+0000 7fa331791700 1 -- 172.21.15.79:0/2775007656 learned_addr learned my addr 172.21.15.79:0/2775007656 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:04.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.407+0000 7fa330f90700 1 --2- 172.21.15.79:0/2775007656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132aff50 0x5572132af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.407+0000 7fa331f92700 1 --2- 172.21.15.79:0/2775007656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132d7d80 0x5572132f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.407+0000 7fa331791700 1 -- 172.21.15.79:0/2775007656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132d7d80 msgr2=0x5572132f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:04.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.407+0000 7fa331791700 1 --2- 172.21.15.79:0/2775007656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132d7d80 0x5572132f1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.407+0000 7fa331791700 1 -- 172.21.15.79:0/2775007656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132aff50 msgr2=0x5572132af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:04.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.407+0000 7fa331791700 1 --2- 172.21.15.79:0/2775007656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132aff50 0x5572132af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.407+0000 7fa331791700 1 -- 172.21.15.79:0/2775007656 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557213017c10 con 0x5572132ad740 2023-06-23T18:37:04.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.407+0000 7fa331791700 1 --2- 172.21.15.79:0/2775007656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132ad740 0x5572132aeef0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0096d0 tx=0x7fa31c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4f4baac0a7e67 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:04.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.407+0000 7fa30f7fe700 1 -- 172.21.15.79:0/2775007656 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c02c320 con 0x5572132ad740 2023-06-23T18:37:04.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.407+0000 7fa30f7fe700 1 -- 172.21.15.79:0/2775007656 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31c02c480 con 0x5572132ad740 2023-06-23T18:37:04.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.407+0000 7fa30f7fe700 1 -- 172.21.15.79:0/2775007656 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c02c610 con 0x5572132ad740 2023-06-23T18:37:04.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.408+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/2775007656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132ad740 msgr2=0x5572132aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.408+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/2775007656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132ad740 0x5572132aeef0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0096d0 tx=0x7fa31c017000 comp rx=0 tx=0).stop 2023-06-23T18:37:04.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.408+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/2775007656 shutdown_connections 2023-06-23T18:37:04.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.408+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/2775007656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132d7d80 0x5572132f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.408+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/2775007656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132ad740 0x5572132aeef0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.408+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/2775007656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132aff50 0x5572132af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.408+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/2775007656 >> 172.21.15.79:0/2775007656 conn(0x5572132a4fc0 msgr2=0x5572132a6490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:04.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.408+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/2775007656 shutdown_connections 2023-06-23T18:37:04.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.408+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/2775007656 wait complete. 2023-06-23T18:37:04.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.408+0000 7fa33fb7ab80 1 -- start start 2023-06-23T18:37:04.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.408+0000 7fa33fb7ab80 1 Processor -- start 2023-06-23T18:37:04.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa33fb7ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132ad740 0x55721335a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa33fb7ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132aff50 0x5572132adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa33fb7ab80 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132d7d80 0x5572132ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa33fb7ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572132aa240 con 0x5572132ad740 2023-06-23T18:37:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa33fb7ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572132f7af0 con 0x5572132aff50 2023-06-23T18:37:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa33fb7ab80 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x55721335ae00 con 0x5572132d7d80 2023-06-23T18:37:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa33fb7ab80 10 client.0 _ll_register_callbacks cb 0x5572131a6210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:37:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa331791700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132ad740 0x55721335a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa331791700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132ad740 0x55721335a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41700/0 (socket says 172.21.15.79:41700) 2023-06-23T18:37:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa331791700 1 -- 172.21.15.79:0/1061615599 learned_addr learned my addr 172.21.15.79:0/1061615599 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa331f92700 1 --2- 172.21.15.79:0/1061615599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132d7d80 0x5572132ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa330f90700 1 --2- 172.21.15.79:0/1061615599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132aff50 0x5572132adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa331791700 1 -- 172.21.15.79:0/1061615599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132d7d80 msgr2=0x5572132ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa331791700 1 --2- 172.21.15.79:0/1061615599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132d7d80 0x5572132ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa331791700 1 -- 172.21.15.79:0/1061615599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132aff50 msgr2=0x5572132adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa331791700 1 --2- 172.21.15.79:0/1061615599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132aff50 0x5572132adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa331791700 1 -- 172.21.15.79:0/1061615599 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c014da0 con 0x5572132ad740 2023-06-23T18:37:04.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.409+0000 7fa330f90700 1 --2- 172.21.15.79:0/1061615599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132aff50 0x5572132adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:04.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa331791700 1 --2- 172.21.15.79:0/1061615599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132ad740 0x55721335a8c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa31c019040 tx=0x7fa31c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb13bc58ad19d068 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:04.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa307fff700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2023-06-23T18:37:04.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa307fff700 1 -- 172.21.15.79:0/1061615599 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c024e10 con 0x5572132ad740 2023-06-23T18:37:04.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa307fff700 1 -- 172.21.15.79:0/1061615599 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa31c029070 con 0x5572132ad740 2023-06-23T18:37:04.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1061615599 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa328003da0 con 0x5572132ad740 2023-06-23T18:37:04.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa307fff700 1 -- 172.21.15.79:0/1061615599 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa31c02c8d0 con 0x5572132ad740 2023-06-23T18:37:04.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa33fb7ab80 10 client.5493 fetch_fsmap learned FSMap version 16 2023-06-23T18:37:04.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1061615599 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x557213055200 con 0x5572132ad740 2023-06-23T18:37:04.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa307fff700 1 -- 172.21.15.79:0/1061615599 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa328003da0 con 0x5572132ad740 2023-06-23T18:37:04.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa307fff700 1 -- 172.21.15.79:0/1061615599 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa31c02ca30 con 0x5572132ad740 2023-06-23T18:37:04.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa33fb7ab80 10 client.5493 fetch_fsmap finished waiting for FSMap version 16 2023-06-23T18:37:04.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa33fb7ab80 10 client.5493 resolve_mds: resolved name 'c' to daemon mds.c 2023-06-23T18:37:04.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa33fb7ab80 20 client.5493 populate_metadata read hostname 'smithi079' 2023-06-23T18:37:04.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1061615599 >> [v2:172.21.15.79:6834/1030786659,v1:172.21.15.79:6836/1030786659] conn(0x5572132a9390 0x5572132a9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa33fb7ab80 4 client.5493 mds_command: new command op to 5292 tid=0[{"prefix": "status"}] 2023-06-23T18:37:04.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1061615599 --> [v2:172.21.15.79:6834/1030786659,v1:172.21.15.79:6836/1030786659] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5572131bc000 con 0x5572132a9390 2023-06-23T18:37:04.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.410+0000 7fa330f90700 1 --2- 172.21.15.79:0/1061615599 >> [v2:172.21.15.79:6834/1030786659,v1:172.21.15.79:6836/1030786659] conn(0x5572132a9390 0x5572132a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa307fff700 10 client.5493 ms_handle_connect on v2:172.21.15.79:6834/1030786659 2023-06-23T18:37:04.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa330f90700 1 --2- 172.21.15.79:0/1061615599 >> [v2:172.21.15.79:6834/1030786659,v1:172.21.15.79:6836/1030786659] conn(0x5572132a9390 0x5572132a9760 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1651ff1069651d1 server_cookie=2740f85b17df06de in_seq=0 out_seq=0 2023-06-23T18:37:04.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa307fff700 1 -- 172.21.15.79:0/1061615599 <== mds.0 v2:172.21.15.79:6834/1030786659 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5572131bc000 con 0x5572132a9390 2023-06-23T18:37:04.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa307fff700 10 client.5493 handle_command_reply: tid=0 2023-06-23T18:37:04.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa33fb7ab80 1 client.5493 shutdown 2023-06-23T18:37:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1061615599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132ad740 msgr2=0x55721335a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1061615599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132ad740 0x55721335a8c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa31c019040 tx=0x7fa31c0047c0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1061615599 shutdown_connections 2023-06-23T18:37:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1061615599 >> [v2:172.21.15.79:6834/1030786659,v1:172.21.15.79:6836/1030786659] conn(0x5572132a9390 0x5572132a9760 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1061615599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5572132d7d80 0x5572132ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa307fff700 0 client.5493 ms_handle_reset on v2:172.21.15.79:6834/1030786659 2023-06-23T18:37:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1061615599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572132ad740 0x55721335a8c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa33fb7ab80 1 --2- 172.21.15.79:0/1061615599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572132aff50 0x5572132adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1061615599 >> 172.21.15.79:0/1061615599 conn(0x5572132a4fc0 msgr2=0x5572132d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:04.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1061615599 shutdown_connections 2023-06-23T18:37:04.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa33fb7ab80 1 -- 172.21.15.79:0/1061615599 wait complete. 2023-06-23T18:37:04.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.411+0000 7fa33fb7ab80 20 client.5493 trim_cache size 0 max 16384 2023-06-23T18:37:04.426 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-23T18:37:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "c6d43825-8b0e-40e8-8d7f-a5ab36592233", 2023-06-23T18:37:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 0, 2023-06-23T18:37:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 5292, 2023-06-23T18:37:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2023-06-23T18:37:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2023-06-23T18:37:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2023-06-23T18:37:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 5.8650095279999999, 2023-06-23T18:37:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 16, 2023-06-23T18:37:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 34, 2023-06-23T18:37:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 34, 2023-06-23T18:37:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 13.116561712999999 2023-06-23T18:37:04.428 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-23T18:37:04.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.412+0000 7fa3339f5700 1 -- 172.21.15.79:0/3933574913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa31006ace0 msgr2=0x7fa31006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.412+0000 7fa3339f5700 1 --2- 172.21.15.79:0/3933574913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa31006ace0 0x7fa31006d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009bd0 tx=0x7fa31c008040 comp rx=0 tx=0).stop 2023-06-23T18:37:04.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.412+0000 7fa3339f5700 1 -- 172.21.15.79:0/3933574913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa32c0b6200 msgr2=0x7fa32c1f16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.412+0000 7fa3339f5700 1 --2- 172.21.15.79:0/3933574913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa32c0b6200 0x7fa32c1f16d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa32800e4b0 tx=0x7fa3280047c0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.412+0000 7fa3339f5700 1 -- 172.21.15.79:0/3933574913 shutdown_connections 2023-06-23T18:37:04.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.412+0000 7fa3339f5700 1 --2- 172.21.15.79:0/3933574913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa32c1e3d90 0x7fa32c1e4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.412+0000 7fa3339f5700 1 --2- 172.21.15.79:0/3933574913 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa31006ace0 0x7fa31006d190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.412+0000 7fa3339f5700 1 --2- 172.21.15.79:0/3933574913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa318003f80 0x7fa32c1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.412+0000 7fa3339f5700 1 --2- 172.21.15.79:0/3933574913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa32c0b6200 0x7fa32c1f16d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.412+0000 7fa3339f5700 1 -- 172.21.15.79:0/3933574913 >> 172.21.15.79:0/3933574913 conn(0x7fa32c089fc0 msgr2=0x7fa32c0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:04.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.412+0000 7fa3339f5700 1 -- 172.21.15.79:0/3933574913 shutdown_connections 2023-06-23T18:37:04.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.412+0000 7fa3339f5700 1 -- 172.21.15.79:0/3933574913 wait complete. 2023-06-23T18:37:04.431 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-23T18:37:04.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.545+0000 7f96cc4cf700 1 Processor -- start 2023-06-23T18:37:04.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.545+0000 7f96cc4cf700 1 -- start start 2023-06-23T18:37:04.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.546+0000 7f96cc4cf700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f96c4071490 0x7f96c4071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.546+0000 7f96cc4cf700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f96c40b1270 0x7f96c4071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.546+0000 7f96cc4cf700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c41608f0 con 0x7f96c40aee30 2023-06-23T18:37:04.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.546+0000 7f96cc4cf700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c40722e0 con 0x7f96c4071490 2023-06-23T18:37:04.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.546+0000 7f96cc4cf700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c4160e20 con 0x7f96c40b1270 2023-06-23T18:37:04.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.546+0000 7f96ca26b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f96c4071490 0x7f96c4071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.546+0000 7f96ca26b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f96c4071490 0x7f96c4071860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41712/0 (socket says 172.21.15.79:41712) 2023-06-23T18:37:04.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.546+0000 7f96ca26b700 1 -- 172.21.15.79:0/2828418499 learned_addr learned my addr 172.21.15.79:0/2828418499 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:04.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.546+0000 7f96ca26b700 1 -- 172.21.15.79:0/2828418499 >> v2:172.21.15.155:3300/0 conn(0x7f96c40b1270 msgr2=0x7f96c4071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:04.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.546+0000 7f96ca26b700 1 --2- 172.21.15.79:0/2828418499 >> v2:172.21.15.155:3300/0 conn(0x7f96c40b1270 0x7f96c4071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.546+0000 7f96ca26b700 1 -- 172.21.15.79:0/2828418499 >> v1:172.21.15.155:6789/0 conn(0x7f96c40aee30 legacy=0x7f96c4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:04.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.546+0000 7f96ca26b700 1 -- 172.21.15.79:0/2828418499 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c40cd4f0 con 0x7f96c4071490 2023-06-23T18:37:04.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.547+0000 7f96ca26b700 1 --2- 172.21.15.79:0/2828418499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96c4071490 0x7f96c4071860 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f96c40cd230 tx=0x7f96b4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd610d14a7cc360 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:04.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.547+0000 7f96c9269700 1 -- 172.21.15.79:0/2828418499 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4004230 con 0x7f96c4071490 2023-06-23T18:37:04.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.547+0000 7f96c9269700 1 -- 172.21.15.79:0/2828418499 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b4004390 con 0x7f96c4071490 2023-06-23T18:37:04.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.547+0000 7f96c9269700 1 -- 172.21.15.79:0/2828418499 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b40036a0 con 0x7f96c4071490 2023-06-23T18:37:04.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.547+0000 7f96cc4cf700 1 -- 172.21.15.79:0/2828418499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96c4071490 msgr2=0x7f96c4071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.547+0000 7f96cc4cf700 1 --2- 172.21.15.79:0/2828418499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96c4071490 0x7f96c4071860 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f96c40cd230 tx=0x7f96b4009bd0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.547+0000 7f96cc4cf700 1 -- 172.21.15.79:0/2828418499 shutdown_connections 2023-06-23T18:37:04.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.547+0000 7f96cc4cf700 1 --2- 172.21.15.79:0/2828418499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96c4071490 0x7f96c4071860 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.547+0000 7f96cc4cf700 1 --2- 172.21.15.79:0/2828418499 >> v2:172.21.15.155:3300/0 conn(0x7f96c40b1270 0x7f96c4071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.547+0000 7f96cc4cf700 1 -- 172.21.15.79:0/2828418499 >> 172.21.15.79:0/2828418499 conn(0x7f96c406d660 msgr2=0x7f96c406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:04.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.547+0000 7f96cc4cf700 1 -- 172.21.15.79:0/2828418499 shutdown_connections 2023-06-23T18:37:04.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.547+0000 7f96cc4cf700 1 -- 172.21.15.79:0/2828418499 wait complete. 2023-06-23T18:37:04.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.549+0000 7f96cc4cf700 1 Processor -- start 2023-06-23T18:37:04.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.549+0000 7f96cc4cf700 1 -- start start 2023-06-23T18:37:04.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.549+0000 7f96cc4cf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96c4071490 0x7f96c415b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.549+0000 7f96cc4cf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96c40aee30 0x7f96c40b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.549+0000 7f96cc4cf700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96c40b1270 0x7f96c414e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.549+0000 7f96cc4cf700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c40b3030 con 0x7f96c4071490 2023-06-23T18:37:04.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.549+0000 7f96cc4cf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c40b5560 con 0x7f96c40aee30 2023-06-23T18:37:04.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.549+0000 7f96cc4cf700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c414e690 con 0x7f96c40b1270 2023-06-23T18:37:04.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.549+0000 7f96ca26b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96c4071490 0x7f96c415b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.549+0000 7f96ca26b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96c4071490 0x7f96c415b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41714/0 (socket says 172.21.15.79:41714) 2023-06-23T18:37:04.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.549+0000 7f96ca26b700 1 -- 172.21.15.79:0/4187700516 learned_addr learned my addr 172.21.15.79:0/4187700516 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:04.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.550+0000 7f96caa6c700 1 --2- 172.21.15.79:0/4187700516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96c40b1270 0x7f96c414e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.550+0000 7f96c9a6a700 1 --2- 172.21.15.79:0/4187700516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96c40aee30 0x7f96c40b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.550+0000 7f96ca26b700 1 -- 172.21.15.79:0/4187700516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96c40b1270 msgr2=0x7f96c414e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.550+0000 7f96ca26b700 1 --2- 172.21.15.79:0/4187700516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96c40b1270 0x7f96c414e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.550+0000 7f96ca26b700 1 -- 172.21.15.79:0/4187700516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96c40aee30 msgr2=0x7f96c40b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.550+0000 7f96ca26b700 1 --2- 172.21.15.79:0/4187700516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96c40aee30 0x7f96c40b5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.550+0000 7f96ca26b700 1 -- 172.21.15.79:0/4187700516 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b4009550 con 0x7f96c4071490 2023-06-23T18:37:04.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.550+0000 7f96ca26b700 1 --2- 172.21.15.79:0/4187700516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96c4071490 0x7f96c415b820 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f96b4000c00 tx=0x7f96b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:04.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.550+0000 7f96baffd700 1 -- 172.21.15.79:0/4187700516 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4011830 con 0x7f96c4071490 2023-06-23T18:37:04.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.550+0000 7f96baffd700 1 -- 172.21.15.79:0/4187700516 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b4011990 con 0x7f96c4071490 2023-06-23T18:37:04.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.550+0000 7f96baffd700 1 -- 172.21.15.79:0/4187700516 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4011d40 con 0x7f96c4071490 2023-06-23T18:37:04.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.550+0000 7f96cc4cf700 1 -- 172.21.15.79:0/4187700516 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c40b3280 con 0x7f96c4071490 2023-06-23T18:37:04.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.550+0000 7f96cc4cf700 1 -- 172.21.15.79:0/4187700516 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c40b3840 con 0x7f96c4071490 2023-06-23T18:37:04.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.551+0000 7f96cc4cf700 1 -- 172.21.15.79:0/4187700516 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c415dbb0 con 0x7f96c4071490 2023-06-23T18:37:04.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.551+0000 7f96baffd700 1 -- 172.21.15.79:0/4187700516 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f96b4024460 con 0x7f96c4071490 2023-06-23T18:37:04.562 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.552+0000 7f96baffd700 1 --2- 172.21.15.79:0/4187700516 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f96a806ace0 0x7f96a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:04.562 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.552+0000 7f96baffd700 1 -- 172.21.15.79:0/4187700516 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f96b408ae20 con 0x7f96c4071490 2023-06-23T18:37:04.562 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.552+0000 7f96c9a6a700 1 --2- 172.21.15.79:0/4187700516 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f96a806ace0 0x7f96a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:04.562 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.552+0000 7f96c9a6a700 1 --2- 172.21.15.79:0/4187700516 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f96a806ace0 0x7f96a806d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f96c40b0b70 tx=0x7f96c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:04.562 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.556+0000 7f96baffd700 1 -- 172.21.15.79:0/4187700516 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f96b4055f60 con 0x7f96c4071490 2023-06-23T18:37:04.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.791+0000 7f96cc4cf700 1 -- 172.21.15.79:0/4187700516 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96c40724a0 con 0x7f96c4071490 2023-06-23T18:37:04.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.791+0000 7f96baffd700 1 -- 172.21.15.79:0/4187700516 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f96b4029060 con 0x7f96c4071490 2023-06-23T18:37:04.797 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-23T18:37:04.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.793+0000 7f96cc4cf700 1 -- 172.21.15.79:0/4187700516 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f96a806ace0 msgr2=0x7f96a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.793+0000 7f96cc4cf700 1 --2- 172.21.15.79:0/4187700516 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f96a806ace0 0x7f96a806d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f96c40b0b70 tx=0x7f96c0008040 comp rx=0 tx=0).stop 2023-06-23T18:37:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.793+0000 7f96cc4cf700 1 -- 172.21.15.79:0/4187700516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96c4071490 msgr2=0x7f96c415b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.793+0000 7f96cc4cf700 1 --2- 172.21.15.79:0/4187700516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96c4071490 0x7f96c415b820 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f96b4000c00 tx=0x7f96b4004750 comp rx=0 tx=0).stop 2023-06-23T18:37:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.793+0000 7f96cc4cf700 1 -- 172.21.15.79:0/4187700516 shutdown_connections 2023-06-23T18:37:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.793+0000 7f96cc4cf700 1 --2- 172.21.15.79:0/4187700516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96c40b1270 0x7f96c414e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.793+0000 7f96cc4cf700 1 --2- 172.21.15.79:0/4187700516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96c4071490 0x7f96c415b820 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.793+0000 7f96cc4cf700 1 --2- 172.21.15.79:0/4187700516 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f96a806ace0 0x7f96a806d190 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.793+0000 7f96cc4cf700 1 --2- 172.21.15.79:0/4187700516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96c40aee30 0x7f96c40b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:04.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.793+0000 7f96cc4cf700 1 -- 172.21.15.79:0/4187700516 >> 172.21.15.79:0/4187700516 conn(0x7f96c406d660 msgr2=0x7f96c40ca780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:04.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.793+0000 7f96cc4cf700 1 -- 172.21.15.79:0/4187700516 shutdown_connections 2023-06-23T18:37:04.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:04.793+0000 7f96cc4cf700 1 -- 172.21.15.79:0/4187700516 wait complete. 2023-06-23T18:37:04.810 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-23T18:37:04.810 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-23T18:37:04.811 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi079.front.sepia.ceph.com 2023-06-23T18:37:04.811 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-06-23T18:37:04.811 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-23T18:37:04.811 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-23T18:37:04.812 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-23T18:37:04.812 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-23T18:37:04.812 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-23T18:37:04.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:04.813 DEBUG:teuthology.orchestra.run.smithi079:> ip addr 2023-06-23T18:37:04.830 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:37:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:37:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:37:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:37:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2023-06-23T18:37:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:37:04.831 INFO:teuthology.orchestra.run.smithi079.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:37:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6a brd ff:ff:ff:ff:ff:ff 2023-06-23T18:37:04.831 INFO:teuthology.orchestra.run.smithi079.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:37:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6b brd ff:ff:ff:ff:ff:ff 2023-06-23T18:37:04.831 INFO:teuthology.orchestra.run.smithi079.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:37:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:86 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:37:04.832 INFO:teuthology.orchestra.run.smithi079.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-23T18:37:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:87 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:37:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: inet 172.21.15.79/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-23T18:37:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft 42356sec preferred_lft 42356sec 2023-06-23T18:37:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::162a:fb46:60c1:412/64 scope link noprefixroute 2023-06-23T18:37:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:37:04.832 INFO:teuthology.orchestra.run.smithi079.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:37:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 5e:7f:c1:73:2d:85 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:37:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-23T18:37:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:37:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::f86d:beff:fe28:e075/64 scope link 2023-06-23T18:37:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:37:04.833 INFO:teuthology.orchestra.run.smithi079.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-23T18:37:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 8a:d5:d7:85:fa:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-23T18:37:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::88d5:d7ff:fe85:fa5d/64 scope link 2023-06-23T18:37:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:37:04.834 INFO:teuthology.orchestra.run.smithi079.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-23T18:37:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 5e:7f:c1:73:2d:85 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-06-23T18:37:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::5c7f:c1ff:fe73:2d85/64 scope link 2023-06-23T18:37:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:37:04.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:04.835 DEBUG:teuthology.orchestra.run.smithi079:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-23T18:37:04.895 INFO:teuthology.orchestra.run.smithi079.stdout:1 2023-06-23T18:37:04.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:04.897 DEBUG:teuthology.orchestra.run.smithi079:> route 2023-06-23T18:37:04.955 INFO:teuthology.orchestra.run.smithi079.stdout:Kernel IP routing table 2023-06-23T18:37:04.955 INFO:teuthology.orchestra.run.smithi079.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-23T18:37:04.955 INFO:teuthology.orchestra.run.smithi079.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-23T18:37:04.955 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-23T18:37:04.956 INFO:teuthology.orchestra.run.smithi079.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-23T18:37:04.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:04.956 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-23T18:37:04.957 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-23T18:37:04.957 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-23T18:37:04.957 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-23T18:37:04.958 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-23T18:37:04.958 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-23T18:37:05.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:05.061 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2023-06-23T18:37:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-06-23T18:37:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-06-23T18:37:05.079 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-23T18:37:05.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:05.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set brx.0 up 2023-06-23T18:37:05.145 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-06-23T18:37:05.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:37:05.146 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-06-23T18:37:05.201 INFO:teuthology.orchestra.run.smithi079.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-06-23T18:37:05.202 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:37:05.202 DEBUG:teuthology.orchestra.run.smithi079:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-06-23T18:37:05.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2023-06-23T18:37:05.325 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:37:05.326 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:37:05.391 INFO:teuthology.orchestra.run.smithi079.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:37:05.392 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:37:05.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:05.395 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2023-06-23T18:37:05.450 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-06-23T18:37:05.451 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-06-23T18:37:05.493 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2023-06-23T18:37:05.520 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:37:05.520 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:37:05.579 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi079.stderr:2023-06-23T18:37:05.574+0000 7fe1cb3c75c0 -1 init, newargv = 0x556b0b07ae80 newargc=15 2023-06-23T18:37:05.579 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi079.stderr:ceph-fuse[116927]: starting ceph client 2023-06-23T18:37:05.585 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi079.stderr:ceph-fuse[116927]: starting fuse 2023-06-23T18:37:05.587 INFO:teuthology.orchestra.run.smithi079.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:37:05.589 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:37:05.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:05.590 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2023-06-23T18:37:05.646 INFO:teuthology.orchestra.run.smithi079.stdout:47 2023-06-23T18:37:05.647 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-06-23T18:37:05.647 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-23T18:37:05.647 DEBUG:teuthology.orchestra.run.smithi079:> import glob 2023-06-23T18:37:05.648 DEBUG:teuthology.orchestra.run.smithi079:> import re 2023-06-23T18:37:05.648 DEBUG:teuthology.orchestra.run.smithi079:> import os 2023-06-23T18:37:05.648 DEBUG:teuthology.orchestra.run.smithi079:> import subprocess 2023-06-23T18:37:05.649 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-23T18:37:05.649 DEBUG:teuthology.orchestra.run.smithi079:> def _find_admin_socket(client_name): 2023-06-23T18:37:05.650 DEBUG:teuthology.orchestra.run.smithi079:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-06-23T18:37:05.650 DEBUG:teuthology.orchestra.run.smithi079:> files = glob.glob(asok_path) 2023-06-23T18:37:05.650 DEBUG:teuthology.orchestra.run.smithi079:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-06-23T18:37:05.650 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-23T18:37:05.651 DEBUG:teuthology.orchestra.run.smithi079:> # Given a non-glob path, it better be there 2023-06-23T18:37:05.651 DEBUG:teuthology.orchestra.run.smithi079:> if "*" not in asok_path: 2023-06-23T18:37:05.651 DEBUG:teuthology.orchestra.run.smithi079:> assert(len(files) == 1) 2023-06-23T18:37:05.652 DEBUG:teuthology.orchestra.run.smithi079:> return files[0] 2023-06-23T18:37:05.652 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-23T18:37:05.652 DEBUG:teuthology.orchestra.run.smithi079:> for f in files: 2023-06-23T18:37:05.653 DEBUG:teuthology.orchestra.run.smithi079:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-23T18:37:05.653 DEBUG:teuthology.orchestra.run.smithi079:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-23T18:37:05.653 DEBUG:teuthology.orchestra.run.smithi079:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-23T18:37:05.653 DEBUG:teuthology.orchestra.run.smithi079:> contents = proc_f.read() 2023-06-23T18:37:05.654 DEBUG:teuthology.orchestra.run.smithi079:> if mountpoint in contents: 2023-06-23T18:37:05.654 DEBUG:teuthology.orchestra.run.smithi079:> return f 2023-06-23T18:37:05.654 DEBUG:teuthology.orchestra.run.smithi079:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-23T18:37:05.655 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-23T18:37:05.655 DEBUG:teuthology.orchestra.run.smithi079:> print(_find_admin_socket("client.0")) 2023-06-23T18:37:05.655 DEBUG:teuthology.orchestra.run.smithi079:> ' 2023-06-23T18:37:05.768 INFO:teuthology.orchestra.run.smithi079.stdout:/var/run/ceph/ceph-client.0.116927.asok 2023-06-23T18:37:05.951 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.116927.asok 2023-06-23T18:37:05.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:05.952 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.116927.asok status 2023-06-23T18:37:06.107 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-23T18:37:06.108 INFO:teuthology.orchestra.run.smithi079.stdout: "metadata": { 2023-06-23T18:37:06.108 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_sha1": "8627af3e0adcb765a3c249fcc209cba9f4873e1b", 2023-06-23T18:37:06.108 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:37:06.108 INFO:teuthology.orchestra.run.smithi079.stdout: "entity_id": "0", 2023-06-23T18:37:06.109 INFO:teuthology.orchestra.run.smithi079.stdout: "hostname": "smithi079", 2023-06-23T18:37:06.109 INFO:teuthology.orchestra.run.smithi079.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-06-23T18:37:06.109 INFO:teuthology.orchestra.run.smithi079.stdout: "pid": "116927", 2023-06-23T18:37:06.110 INFO:teuthology.orchestra.run.smithi079.stdout: "root": "/" 2023-06-23T18:37:06.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:37:06.110 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_count": 0, 2023-06-23T18:37:06.110 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_pinned_count": 0, 2023-06-23T18:37:06.111 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 5505, 2023-06-23T18:37:06.111 INFO:teuthology.orchestra.run.smithi079.stdout: "inst": { 2023-06-23T18:37:06.111 INFO:teuthology.orchestra.run.smithi079.stdout: "name": { 2023-06-23T18:37:06.111 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "client", 2023-06-23T18:37:06.112 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 5505 2023-06-23T18:37:06.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:37:06.112 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2023-06-23T18:37:06.113 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-06-23T18:37:06.113 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:37:06.113 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2074877102 2023-06-23T18:37:06.113 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-23T18:37:06.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:37:06.114 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2023-06-23T18:37:06.114 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-06-23T18:37:06.114 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:37:06.114 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2074877102 2023-06-23T18:37:06.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-23T18:37:06.115 INFO:teuthology.orchestra.run.smithi079.stdout: "inst_str": "client.5505 192.168.0.1:0/2074877102", 2023-06-23T18:37:06.115 INFO:teuthology.orchestra.run.smithi079.stdout: "addr_str": "192.168.0.1:0/2074877102", 2023-06-23T18:37:06.115 INFO:teuthology.orchestra.run.smithi079.stdout: "inode_count": 1, 2023-06-23T18:37:06.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_epoch": 16, 2023-06-23T18:37:06.115 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch": 34, 2023-06-23T18:37:06.115 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch_barrier": 0, 2023-06-23T18:37:06.116 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklisted": false, 2023-06-23T18:37:06.116 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs" 2023-06-23T18:37:06.116 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-23T18:37:06.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:06.117 DEBUG:teuthology.orchestra.run.smithi079:> stat --file-system '--printf=%T 2023-06-23T18:37:06.118 DEBUG:teuthology.orchestra.run.smithi079:> ' -- /home/ubuntu/cephtest/mnt.0 2023-06-23T18:37:06.135 INFO:teuthology.orchestra.run.smithi079.stdout:fuseblk 2023-06-23T18:37:06.135 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-06-23T18:37:06.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:06.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-06-23T18:37:06.201 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-23T18:37:06.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.357+0000 7f53c46d8700 1 Processor -- start 2023-06-23T18:37:06.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.357+0000 7f53c46d8700 1 -- start start 2023-06-23T18:37:06.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.358+0000 7f53c46d8700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f53bc15fa60 0x7f53bc0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:06.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.358+0000 7f53c46d8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f53bc1609f0 0x7f53bc0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:06.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.358+0000 7f53c46d8700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53bc160de0 con 0x7f53bc0b17d0 2023-06-23T18:37:06.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.358+0000 7f53c46d8700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53bc15fe30 con 0x7f53bc1609f0 2023-06-23T18:37:06.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.358+0000 7f53c46d8700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53bc06ce10 con 0x7f53bc15fa60 2023-06-23T18:37:06.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.358+0000 7f53c2c75700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f53bc1609f0 0x7f53bc0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:06.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.358+0000 7f53c2c75700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f53bc1609f0 0x7f53bc0cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41740/0 (socket says 172.21.15.79:41740) 2023-06-23T18:37:06.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.358+0000 7f53c2c75700 1 -- 172.21.15.79:0/324987258 learned_addr learned my addr 172.21.15.79:0/324987258 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:06.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.359+0000 7f53c2474700 1 --2- 172.21.15.79:0/324987258 >> v2:172.21.15.155:3300/0 conn(0x7f53bc15fa60 0x7f53bc0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:06.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.359+0000 7f53c2c75700 1 -- 172.21.15.79:0/324987258 >> v2:172.21.15.155:3300/0 conn(0x7f53bc15fa60 msgr2=0x7f53bc0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:06.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.359+0000 7f53c2c75700 1 --2- 172.21.15.79:0/324987258 >> v2:172.21.15.155:3300/0 conn(0x7f53bc15fa60 0x7f53bc0ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:06.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.359+0000 7f53c2c75700 1 -- 172.21.15.79:0/324987258 >> v1:172.21.15.155:6789/0 conn(0x7f53bc0b17d0 legacy=0x7f53bc1602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:06.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.359+0000 7f53c2c75700 1 -- 172.21.15.79:0/324987258 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc0b0a10 con 0x7f53bc1609f0 2023-06-23T18:37:06.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.359+0000 7f53c2c75700 1 --2- 172.21.15.79:0/324987258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53bc1609f0 0x7f53bc0cea30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f53bc072440 tx=0x7f53b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e46bb5af28bccf server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:06.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.359+0000 7f53c1472700 1 -- 172.21.15.79:0/324987258 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b400e070 con 0x7f53bc1609f0 2023-06-23T18:37:06.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.359+0000 7f53c1472700 1 -- 172.21.15.79:0/324987258 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b400b910 con 0x7f53bc1609f0 2023-06-23T18:37:06.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.359+0000 7f53c1472700 1 -- 172.21.15.79:0/324987258 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b400bcc0 con 0x7f53bc1609f0 2023-06-23T18:37:06.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.359+0000 7f53c46d8700 1 -- 172.21.15.79:0/324987258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53bc1609f0 msgr2=0x7f53bc0cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:06.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.360+0000 7f53c46d8700 1 --2- 172.21.15.79:0/324987258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53bc1609f0 0x7f53bc0cea30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f53bc072440 tx=0x7f53b400b330 comp rx=0 tx=0).stop 2023-06-23T18:37:06.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.360+0000 7f53c46d8700 1 -- 172.21.15.79:0/324987258 shutdown_connections 2023-06-23T18:37:06.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.360+0000 7f53c46d8700 1 --2- 172.21.15.79:0/324987258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53bc1609f0 0x7f53bc0cea30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:06.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.360+0000 7f53c46d8700 1 --2- 172.21.15.79:0/324987258 >> v2:172.21.15.155:3300/0 conn(0x7f53bc15fa60 0x7f53bc0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:06.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.360+0000 7f53c46d8700 1 -- 172.21.15.79:0/324987258 >> 172.21.15.79:0/324987258 conn(0x7f53bc089fc0 msgr2=0x7f53bc099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:06.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.360+0000 7f53c46d8700 1 -- 172.21.15.79:0/324987258 shutdown_connections 2023-06-23T18:37:06.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.360+0000 7f53c46d8700 1 -- 172.21.15.79:0/324987258 wait complete. 2023-06-23T18:37:06.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.361+0000 7f53c46d8700 1 Processor -- start 2023-06-23T18:37:06.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.361+0000 7f53c46d8700 1 -- start start 2023-06-23T18:37:06.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c46d8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53bc0b17d0 0x7f53bc0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:06.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c46d8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53bc15fa60 0x7f53bc14b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:06.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c46d8700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53bc1609f0 0x7f53bc1f0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:06.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c46d8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc0cd580 con 0x7f53bc0b17d0 2023-06-23T18:37:06.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c46d8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc0cd860 con 0x7f53bc15fa60 2023-06-23T18:37:06.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c46d8700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc0cdb40 con 0x7f53bc1609f0 2023-06-23T18:37:06.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c2474700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53bc0b17d0 0x7f53bc0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:06.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c2474700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53bc0b17d0 0x7f53bc0ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41742/0 (socket says 172.21.15.79:41742) 2023-06-23T18:37:06.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c2474700 1 -- 172.21.15.79:0/599490999 learned_addr learned my addr 172.21.15.79:0/599490999 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:06.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c2474700 1 -- 172.21.15.79:0/599490999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53bc1609f0 msgr2=0x7f53bc1f0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:06.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c2474700 1 --2- 172.21.15.79:0/599490999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53bc1609f0 0x7f53bc1f0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:06.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c2474700 1 -- 172.21.15.79:0/599490999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53bc15fa60 msgr2=0x7f53bc14b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:06.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c1c73700 1 --2- 172.21.15.79:0/599490999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53bc15fa60 0x7f53bc14b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:06.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c2474700 1 --2- 172.21.15.79:0/599490999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53bc15fa60 0x7f53bc14b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:06.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c2474700 1 -- 172.21.15.79:0/599490999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b4009b30 con 0x7f53bc0b17d0 2023-06-23T18:37:06.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c2474700 1 --2- 172.21.15.79:0/599490999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53bc0b17d0 0x7f53bc0ccd50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00e990 tx=0x7f53ac00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:06.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53b2ffd700 1 -- 172.21.15.79:0/599490999 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac0096f0 con 0x7f53bc0b17d0 2023-06-23T18:37:06.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53b2ffd700 1 -- 172.21.15.79:0/599490999 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac00c850 con 0x7f53bc0b17d0 2023-06-23T18:37:06.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c46d8700 1 -- 172.21.15.79:0/599490999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53bc0cde20 con 0x7f53bc0b17d0 2023-06-23T18:37:06.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53b2ffd700 1 -- 172.21.15.79:0/599490999 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac00cbc0 con 0x7f53bc0b17d0 2023-06-23T18:37:06.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.362+0000 7f53c46d8700 1 -- 172.21.15.79:0/599490999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc14b9c0 con 0x7f53bc0b17d0 2023-06-23T18:37:06.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.363+0000 7f53c46d8700 1 -- 172.21.15.79:0/599490999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53bc0ce040 con 0x7f53bc0b17d0 2023-06-23T18:37:06.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.364+0000 7f53b2ffd700 1 -- 172.21.15.79:0/599490999 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f53ac00cd20 con 0x7f53bc0b17d0 2023-06-23T18:37:06.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.364+0000 7f53b2ffd700 1 --2- 172.21.15.79:0/599490999 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f53a006ace0 0x7f53a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:06.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.364+0000 7f53b2ffd700 1 -- 172.21.15.79:0/599490999 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f53ac0898d0 con 0x7f53bc0b17d0 2023-06-23T18:37:06.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.364+0000 7f53c1c73700 1 --2- 172.21.15.79:0/599490999 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f53a006ace0 0x7f53a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:06.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.365+0000 7f53c1c73700 1 --2- 172.21.15.79:0/599490999 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f53a006ace0 0x7f53a006d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f53bc152670 tx=0x7f53b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:06.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.369+0000 7f53b2ffd700 1 -- 172.21.15.79:0/599490999 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f53ac054a10 con 0x7f53bc0b17d0 2023-06-23T18:37:06.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.600+0000 7f53c46d8700 1 -- 172.21.15.79:0/599490999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f53bc0cd9a0 con 0x7f53bc0b17d0 2023-06-23T18:37:06.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.601+0000 7f53b2ffd700 1 -- 172.21.15.79:0/599490999 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f53ac021060 con 0x7f53bc0b17d0 2023-06-23T18:37:06.606 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-23T18:37:06.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.603+0000 7f53c46d8700 1 -- 172.21.15.79:0/599490999 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f53a006ace0 msgr2=0x7f53a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:06.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.603+0000 7f53c46d8700 1 --2- 172.21.15.79:0/599490999 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f53a006ace0 0x7f53a006d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f53bc152670 tx=0x7f53b8009040 comp rx=0 tx=0).stop 2023-06-23T18:37:06.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.603+0000 7f53c46d8700 1 -- 172.21.15.79:0/599490999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53bc0b17d0 msgr2=0x7f53bc0ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:06.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.603+0000 7f53c46d8700 1 --2- 172.21.15.79:0/599490999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53bc0b17d0 0x7f53bc0ccd50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00e990 tx=0x7f53ac00ed50 comp rx=0 tx=0).stop 2023-06-23T18:37:06.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.603+0000 7f53c46d8700 1 -- 172.21.15.79:0/599490999 shutdown_connections 2023-06-23T18:37:06.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.603+0000 7f53c46d8700 1 --2- 172.21.15.79:0/599490999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53bc1609f0 0x7f53bc1f0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:06.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.603+0000 7f53c46d8700 1 --2- 172.21.15.79:0/599490999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53bc0b17d0 0x7f53bc0ccd50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:06.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.603+0000 7f53c46d8700 1 --2- 172.21.15.79:0/599490999 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f53a006ace0 0x7f53a006d190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:06.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.603+0000 7f53c46d8700 1 --2- 172.21.15.79:0/599490999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53bc15fa60 0x7f53bc14b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:06.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.603+0000 7f53c46d8700 1 -- 172.21.15.79:0/599490999 >> 172.21.15.79:0/599490999 conn(0x7f53bc089fc0 msgr2=0x7f53bc06fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:06.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.603+0000 7f53c46d8700 1 -- 172.21.15.79:0/599490999 shutdown_connections 2023-06-23T18:37:06.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:06.603+0000 7f53c46d8700 1 -- 172.21.15.79:0/599490999 wait complete. 2023-06-23T18:37:06.619 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-23T18:37:06.620 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-23T18:37:06.620 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-06-23T18:37:06.621 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-06-23T18:37:06.621 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-06-23T18:37:06.621 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-23T18:37:06.621 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-23T18:37:06.622 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-23T18:37:06.622 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-06-23T18:37:06.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:06.623 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-06-23T18:37:06.644 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:37:06.645 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:37:06.645 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:37:06.645 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:37:06.645 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-06-23T18:37:06.645 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:37:06.645 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:37:06.646 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-06-23T18:37:06.646 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:37:06.646 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-06-23T18:37:06.646 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:37:06.646 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-06-23T18:37:06.646 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-23T18:37:06.646 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-06-23T18:37:06.646 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-23T18:37:06.647 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42361sec preferred_lft 42361sec 2023-06-23T18:37:06.647 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::1cf2:57d9:5266:4005/64 scope link noprefixroute 2023-06-23T18:37:06.647 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:37:06.647 INFO:teuthology.orchestra.run.smithi155.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:37:06.647 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 56:b8:22:e6:c3:1d brd ff:ff:ff:ff:ff:ff 2023-06-23T18:37:06.647 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-23T18:37:06.647 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:37:06.648 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::74fc:c7ff:fe1c:d1f2/64 scope link 2023-06-23T18:37:06.648 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:37:06.648 INFO:teuthology.orchestra.run.smithi155.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-23T18:37:06.648 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether f6:b7:da:f4:04:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-06-23T18:37:06.648 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::f4b7:daff:fef4:416/64 scope link 2023-06-23T18:37:06.648 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:37:06.648 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-23T18:37:06.648 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 56:b8:22:e6:c3:1d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-06-23T18:37:06.649 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::54b8:22ff:fee6:c31d/64 scope link 2023-06-23T18:37:06.649 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:37:06.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:06.650 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-23T18:37:06.715 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-06-23T18:37:06.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:06.717 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-06-23T18:37:06.778 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-06-23T18:37:06.778 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-23T18:37:06.778 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-23T18:37:06.778 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-23T18:37:06.778 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-23T18:37:06.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:06.779 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-06-23T18:37:06.779 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-06-23T18:37:06.780 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-23T18:37:06.780 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-23T18:37:06.780 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-23T18:37:06.781 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-06-23T18:37:06.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:06.890 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-06-23T18:37:06.907 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-06-23T18:37:06.907 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-06-23T18:37:06.907 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-06-23T18:37:06.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:06.908 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2023-06-23T18:37:06.975 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-06-23T18:37:06.976 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:37:06.976 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-06-23T18:37:07.032 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-06-23T18:37:07.032 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:37:07.033 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-06-23T18:37:07.088 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-06-23T18:37:07.155 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:37:07.155 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:37:07.221 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:37:07.222 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:37:07.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:07.223 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-06-23T18:37:07.280 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-06-23T18:37:07.280 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-06-23T18:37:07.322 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-06-23T18:37:07.349 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:37:07.349 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:37:07.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2023-06-23T18:37:07.405+0000 7efe96b6d5c0 -1 init, newargv = 0x55fbaa2f7eb0 newargc=15 2023-06-23T18:37:07.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[108822]: starting ceph client 2023-06-23T18:37:07.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[108822]: starting fuse 2023-06-23T18:37:07.417 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:37:07.418 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:37:07.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:07.419 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-06-23T18:37:07.476 INFO:teuthology.orchestra.run.smithi155.stdout:46 2023-06-23T18:37:07.476 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-06-23T18:37:07.476 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-23T18:37:07.477 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-06-23T18:37:07.477 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-06-23T18:37:07.477 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-06-23T18:37:07.478 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-06-23T18:37:07.478 DEBUG:teuthology.orchestra.run.smithi155:> 2023-06-23T18:37:07.478 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-06-23T18:37:07.479 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-06-23T18:37:07.479 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-06-23T18:37:07.479 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-06-23T18:37:07.479 DEBUG:teuthology.orchestra.run.smithi155:> 2023-06-23T18:37:07.480 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-06-23T18:37:07.480 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-06-23T18:37:07.480 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-06-23T18:37:07.481 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-06-23T18:37:07.481 DEBUG:teuthology.orchestra.run.smithi155:> 2023-06-23T18:37:07.481 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-06-23T18:37:07.481 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-23T18:37:07.482 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-23T18:37:07.482 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-23T18:37:07.482 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-06-23T18:37:07.483 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-06-23T18:37:07.483 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-06-23T18:37:07.483 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-23T18:37:07.483 DEBUG:teuthology.orchestra.run.smithi155:> 2023-06-23T18:37:07.484 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2023-06-23T18:37:07.484 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-06-23T18:37:07.599 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.108822.asok 2023-06-23T18:37:07.781 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.108822.asok 2023-06-23T18:37:07.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:07.782 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.108822.asok status 2023-06-23T18:37:07.937 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-06-23T18:37:07.937 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-06-23T18:37:07.938 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "8627af3e0adcb765a3c249fcc209cba9f4873e1b", 2023-06-23T18:37:07.938 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:37:07.938 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2023-06-23T18:37:07.938 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-06-23T18:37:07.938 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-06-23T18:37:07.938 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "108822", 2023-06-23T18:37:07.938 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-06-23T18:37:07.939 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-06-23T18:37:07.939 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-06-23T18:37:07.939 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-06-23T18:37:07.939 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 4907, 2023-06-23T18:37:07.939 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-06-23T18:37:07.939 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-06-23T18:37:07.939 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-06-23T18:37:07.939 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 4907 2023-06-23T18:37:07.940 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-06-23T18:37:07.940 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-06-23T18:37:07.940 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-06-23T18:37:07.940 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:37:07.940 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 3965450137 2023-06-23T18:37:07.940 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-06-23T18:37:07.941 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-06-23T18:37:07.941 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-06-23T18:37:07.941 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-06-23T18:37:07.941 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:37:07.941 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 3965450137 2023-06-23T18:37:07.941 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-06-23T18:37:07.941 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.4907 192.168.0.1:0/3965450137", 2023-06-23T18:37:07.941 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/3965450137", 2023-06-23T18:37:07.942 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-06-23T18:37:07.942 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 16, 2023-06-23T18:37:07.942 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 34, 2023-06-23T18:37:07.942 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-06-23T18:37:07.942 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-06-23T18:37:07.942 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-06-23T18:37:07.943 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-06-23T18:37:07.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:07.948 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-06-23T18:37:07.949 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2023-06-23T18:37:07.966 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-06-23T18:37:07.967 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-06-23T18:37:07.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:07.968 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-06-23T18:37:08.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:08.033 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-06-23T18:37:08.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:08.059 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 0 ./subdir) 2023-06-23T18:37:08.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:08.130 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes ./subdir) 2023-06-23T18:37:08.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:37:08.190 INFO:teuthology.orchestra.run.smithi079.stderr:./subdir: ceph.quota.max_bytes: No such attribute 2023-06-23T18:37:08.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:08.191 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir) 2023-06-23T18:37:08.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:37:08.250 INFO:teuthology.orchestra.run.smithi079.stderr:setfattr: ./subdir: Invalid argument 2023-06-23T18:37:08.250 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-06-23T18:37:08.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-06-23T18:37:08.335 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-06-23T18:37:08.379 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-06-23T18:37:08.380 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:37:08.380 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:37:08.399 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:37:08.566 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.561+0000 7fda94a11700 1 Processor -- start 2023-06-23T18:37:08.566 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.561+0000 7fda94a11700 1 -- start start 2023-06-23T18:37:08.566 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.562+0000 7fda94a11700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fda900b1270 0x7fda9015c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:08.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.562+0000 7fda94a11700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda900aee30 0x7fda9015cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:08.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.562+0000 7fda94a11700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda901608f0 con 0x7fda90071490 2023-06-23T18:37:08.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.562+0000 7fda94a11700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda90160e20 con 0x7fda900b1270 2023-06-23T18:37:08.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.562+0000 7fda94a11700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda900cd4f0 con 0x7fda900aee30 2023-06-23T18:37:08.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.562+0000 7fda8ed9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda900aee30 0x7fda9015cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:08.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.562+0000 7fda8dd9b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fda900b1270 0x7fda9015c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:08.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.562+0000 7fda8ed9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda900aee30 0x7fda9015cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:34520/0 (socket says 172.21.15.79:34520) 2023-06-23T18:37:08.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.562+0000 7fda8ed9d700 1 -- 172.21.15.79:0/3549618250 learned_addr learned my addr 172.21.15.79:0/3549618250 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:08.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.562+0000 7fda8e59c700 1 --1- 172.21.15.79:0/3549618250 >> v1:172.21.15.155:6789/0 conn(0x7fda90071490 0x7fda90071860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:53060/0 (socket says 172.21.15.79:53060) 2023-06-23T18:37:08.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.563+0000 7fda8dd9b700 1 -- 172.21.15.79:0/3549618250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda900aee30 msgr2=0x7fda9015cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:08.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.563+0000 7fda8dd9b700 1 --2- 172.21.15.79:0/3549618250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda900aee30 0x7fda9015cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:08.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.563+0000 7fda8dd9b700 1 -- 172.21.15.79:0/3549618250 >> v1:172.21.15.155:6789/0 conn(0x7fda90071490 legacy=0x7fda90071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:08.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.563+0000 7fda8dd9b700 1 -- 172.21.15.79:0/3549618250 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda90160b40 con 0x7fda900b1270 2023-06-23T18:37:08.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.563+0000 7fda8d59a700 1 -- 172.21.15.79:0/3549618250 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805622049 0 0) 0x7fda901608f0 con 0x7fda90071490 2023-06-23T18:37:08.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.563+0000 7fda8ed9d700 1 --2- 172.21.15.79:0/3549618250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda900aee30 0x7fda9015cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:08.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.563+0000 7fda8dd9b700 1 --2- 172.21.15.79:0/3549618250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda900b1270 0x7fda9015c9f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fda80009840 tx=0x7fda80009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb8f291ff971176c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:08.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.563+0000 7fda8d59a700 1 -- 172.21.15.79:0/3549618250 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80011830 con 0x7fda900b1270 2023-06-23T18:37:08.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.563+0000 7fda8d59a700 1 -- 172.21.15.79:0/3549618250 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda80011990 con 0x7fda900b1270 2023-06-23T18:37:08.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.563+0000 7fda8d59a700 1 -- 172.21.15.79:0/3549618250 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80011d20 con 0x7fda900b1270 2023-06-23T18:37:08.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.563+0000 7fda94a11700 1 -- 172.21.15.79:0/3549618250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda900b1270 msgr2=0x7fda9015c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:08.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.563+0000 7fda94a11700 1 --2- 172.21.15.79:0/3549618250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda900b1270 0x7fda9015c9f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fda80009840 tx=0x7fda80009b50 comp rx=0 tx=0).stop 2023-06-23T18:37:08.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.564+0000 7fda94a11700 1 -- 172.21.15.79:0/3549618250 shutdown_connections 2023-06-23T18:37:08.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.564+0000 7fda94a11700 1 --2- 172.21.15.79:0/3549618250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda900aee30 0x7fda9015cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:08.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.564+0000 7fda94a11700 1 --2- 172.21.15.79:0/3549618250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda900b1270 0x7fda9015c9f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:08.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.564+0000 7fda94a11700 1 -- 172.21.15.79:0/3549618250 >> 172.21.15.79:0/3549618250 conn(0x7fda9006d660 msgr2=0x7fda9006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:08.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.564+0000 7fda94a11700 1 -- 172.21.15.79:0/3549618250 shutdown_connections 2023-06-23T18:37:08.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.564+0000 7fda94a11700 1 -- 172.21.15.79:0/3549618250 wait complete. 2023-06-23T18:37:08.573 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.565+0000 7fda94a11700 1 Processor -- start 2023-06-23T18:37:08.573 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.565+0000 7fda94a11700 1 -- start start 2023-06-23T18:37:08.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda94a11700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda90071490 0x7fda900ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:08.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda94a11700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda900aee30 0x7fda9014ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:08.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda94a11700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda900b1270 0x7fda900c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:08.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda94a11700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda900c3ff0 con 0x7fda90071490 2023-06-23T18:37:08.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda94a11700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9014da20 con 0x7fda900aee30 2023-06-23T18:37:08.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda94a11700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda900c4290 con 0x7fda900b1270 2023-06-23T18:37:08.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda8e59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda90071490 0x7fda900ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:08.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda8e59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda90071490 0x7fda900ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41756/0 (socket says 172.21.15.79:41756) 2023-06-23T18:37:08.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda8e59c700 1 -- 172.21.15.79:0/4090026027 learned_addr learned my addr 172.21.15.79:0/4090026027 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:08.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda8ed9d700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda900b1270 0x7fda900c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:08.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda8dd9b700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda900aee30 0x7fda9014ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:08.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda8e59c700 1 -- 172.21.15.79:0/4090026027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda900b1270 msgr2=0x7fda900c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:08.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda8e59c700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda900b1270 0x7fda900c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:08.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda8e59c700 1 -- 172.21.15.79:0/4090026027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda900aee30 msgr2=0x7fda9014ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:08.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda8e59c700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda900aee30 0x7fda9014ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:08.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda8e59c700 1 -- 172.21.15.79:0/4090026027 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda88009b30 con 0x7fda90071490 2023-06-23T18:37:08.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda8dd9b700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda900aee30 0x7fda9014ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:08.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda8e59c700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda90071490 0x7fda900ab8b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fda8400c550 tx=0x7fda8400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:08.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda7effd700 1 -- 172.21.15.79:0/4090026027 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84004c80 con 0x7fda90071490 2023-06-23T18:37:08.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda7effd700 1 -- 172.21.15.79:0/4090026027 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda84004e00 con 0x7fda90071490 2023-06-23T18:37:08.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda94a11700 1 -- 172.21.15.79:0/4090026027 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda800094a0 con 0x7fda90071490 2023-06-23T18:37:08.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda7effd700 1 -- 172.21.15.79:0/4090026027 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84007310 con 0x7fda90071490 2023-06-23T18:37:08.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.566+0000 7fda94a11700 1 -- 172.21.15.79:0/4090026027 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda900c4850 con 0x7fda90071490 2023-06-23T18:37:08.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.567+0000 7fda94a11700 1 -- 172.21.15.79:0/4090026027 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda900724a0 con 0x7fda90071490 2023-06-23T18:37:08.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.568+0000 7fda7effd700 1 -- 172.21.15.79:0/4090026027 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda84007470 con 0x7fda90071490 2023-06-23T18:37:08.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.568+0000 7fda7effd700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fda7006ace0 0x7fda7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:08.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.568+0000 7fda8dd9b700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fda7006ace0 0x7fda7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:08.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.568+0000 7fda7effd700 1 -- 172.21.15.79:0/4090026027 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fda84089180 con 0x7fda90071490 2023-06-23T18:37:08.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.569+0000 7fda8dd9b700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fda7006ace0 0x7fda7006d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fda80009840 tx=0x7fda80008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:08.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.573+0000 7fda7effd700 1 -- 172.21.15.79:0/4090026027 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fda84054240 con 0x7fda90071490 2023-06-23T18:37:08.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.764+0000 7fda94a11700 1 -- 172.21.15.79:0/4090026027 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda900b5770 con 0x7fda90071490 2023-06-23T18:37:08.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.765+0000 7fda7effd700 1 -- 172.21.15.79:0/4090026027 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+18305 (secure 0 0 0) 0x7fda84057860 con 0x7fda90071490 2023-06-23T18:37:08.771 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:08.771 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":34,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:36:59.547297+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:36:56.964534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-23T18:36:57.507393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:37:08.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.767+0000 7fda94a11700 1 -- 172.21.15.79:0/4090026027 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fda7006ace0 msgr2=0x7fda7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:08.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.767+0000 7fda94a11700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fda7006ace0 0x7fda7006d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fda80009840 tx=0x7fda80008030 comp rx=0 tx=0).stop 2023-06-23T18:37:08.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.767+0000 7fda94a11700 1 -- 172.21.15.79:0/4090026027 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda90071490 msgr2=0x7fda900ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:08.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.767+0000 7fda94a11700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda90071490 0x7fda900ab8b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fda8400c550 tx=0x7fda8400c910 comp rx=0 tx=0).stop 2023-06-23T18:37:08.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.767+0000 7fda94a11700 1 -- 172.21.15.79:0/4090026027 shutdown_connections 2023-06-23T18:37:08.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.767+0000 7fda94a11700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda900b1270 0x7fda900c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:08.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.767+0000 7fda94a11700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda90071490 0x7fda900ab8b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:08.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.767+0000 7fda94a11700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fda7006ace0 0x7fda7006d190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:08.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.767+0000 7fda94a11700 1 --2- 172.21.15.79:0/4090026027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda900aee30 0x7fda9014ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:08.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.767+0000 7fda94a11700 1 -- 172.21.15.79:0/4090026027 >> 172.21.15.79:0/4090026027 conn(0x7fda9006d660 msgr2=0x7fda9015a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:08.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.767+0000 7fda94a11700 1 -- 172.21.15.79:0/4090026027 shutdown_connections 2023-06-23T18:37:08.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.767+0000 7fda94a11700 1 -- 172.21.15.79:0/4090026027 wait complete. 2023-06-23T18:37:08.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:36:56.964534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:36:57.507393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:37:08.785 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:37:08.908 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.903+0000 7f3c9beaa700 1 Processor -- start 2023-06-23T18:37:08.908 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.903+0000 7f3c9beaa700 1 -- start start 2023-06-23T18:37:08.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c9beaa700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c94156ae0 0x7f3c9415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:08.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c9beaa700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9415daf0 con 0x7f3c940cc4d0 2023-06-23T18:37:08.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c9beaa700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9415a560 con 0x7f3c940cc060 2023-06-23T18:37:08.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c9beaa700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c94072530 con 0x7f3c94156ae0 2023-06-23T18:37:08.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c9a447700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3c940cc4d0 0x7f3c94071e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44810/0 (socket says 172.21.15.79:44810) 2023-06-23T18:37:08.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c9a447700 1 -- 172.21.15.79:0/2218796832 learned_addr learned my addr 172.21.15.79:0/2218796832 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:08.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c99c46700 1 --2- 172.21.15.79:0/2218796832 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c94156ae0 0x7f3c9415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:08.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119554847 0 0) 0x7f3c9415daf0 con 0x7f3c940cc4d0 2023-06-23T18:37:08.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c800035b0 con 0x7f3c940cc4d0 2023-06-23T18:37:08.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715700875 0 0) 0x7f3c9415a560 con 0x7f3c940cc060 2023-06-23T18:37:08.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c9415daf0 con 0x7f3c940cc060 2023-06-23T18:37:08.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 843558969 0 0) 0x7f3c800035b0 con 0x7f3c940cc4d0 2023-06-23T18:37:08.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c94156ae0 msgr2=0x7f3c9415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:08.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c98c44700 1 --2- 172.21.15.79:0/2218796832 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c94156ae0 0x7f3c9415d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:08.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 >> v1:172.21.15.155:6789/0 conn(0x7f3c940cc060 legacy=0x7f3c94071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:08.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c94159950 con 0x7f3c940cc4d0 2023-06-23T18:37:08.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f3c8c0019f0 con 0x7f3c940cc4d0 2023-06-23T18:37:08.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 >> v1:172.21.15.79:6789/0 conn(0x7f3c940cc4d0 legacy=0x7f3c94071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:08.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.904+0000 7f3c98c44700 1 --2- 172.21.15.79:0/2218796832 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c94156ae0 0x7f3c80003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:08.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c98c44700 1 --2- 172.21.15.79:0/2218796832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c80004120 0x7f3c80004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:08.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c98c44700 1 --2- 172.21.15.79:0/2218796832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c80004d90 0x7f3c80005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:08.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8000bac0 con 0x7f3c80004120 2023-06-23T18:37:08.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8000bc00 con 0x7f3c80004d90 2023-06-23T18:37:08.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8000bdc0 con 0x7f3c94156ae0 2023-06-23T18:37:08.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c99445700 1 --2- 172.21.15.79:0/2218796832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c80004120 0x7f3c80004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:08.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c9a447700 1 --2- 172.21.15.79:0/2218796832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c80004d90 0x7f3c80005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:08.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c99c46700 1 --2- 172.21.15.79:0/2218796832 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c94156ae0 0x7f3c80003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:08.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c99445700 1 -- 172.21.15.79:0/2218796832 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c94156ae0 msgr2=0x7f3c80003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:08.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c99445700 1 --2- 172.21.15.79:0/2218796832 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c94156ae0 0x7f3c80003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:08.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c99445700 1 -- 172.21.15.79:0/2218796832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c80004d90 msgr2=0x7f3c80005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:08.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c99445700 1 --2- 172.21.15.79:0/2218796832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c80004d90 0x7f3c80005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:08.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c99445700 1 -- 172.21.15.79:0/2218796832 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c800035b0 con 0x7f3c80004120 2023-06-23T18:37:08.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c99445700 1 --2- 172.21.15.79:0/2218796832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c80004120 0x7f3c80004850 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3c84007fd0 tx=0x7f3c80005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a55ea7f73d760cc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:08.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c84001d80 con 0x7f3c80004120 2023-06-23T18:37:08.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c840052d0 con 0x7f3c80004120 2023-06-23T18:37:08.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.905+0000 7f3c98c44700 1 -- 172.21.15.79:0/2218796832 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c84001470 con 0x7f3c80004120 2023-06-23T18:37:08.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.906+0000 7f3c9beaa700 1 -- 172.21.15.79:0/2218796832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c80004120 msgr2=0x7f3c80004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:08.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.906+0000 7f3c9beaa700 1 --2- 172.21.15.79:0/2218796832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c80004120 0x7f3c80004850 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3c84007fd0 tx=0x7f3c80005880 comp rx=0 tx=0).stop 2023-06-23T18:37:08.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.906+0000 7f3c9a447700 1 -- 172.21.15.79:0/2218796832 reap_dead start 2023-06-23T18:37:08.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.906+0000 7f3c9beaa700 1 -- 172.21.15.79:0/2218796832 shutdown_connections 2023-06-23T18:37:08.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.906+0000 7f3c9beaa700 1 -- 172.21.15.79:0/2218796832 >> 172.21.15.79:0/2218796832 conn(0x7f3c9406afa0 msgr2=0x7f3c9406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:08.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.906+0000 7f3c9beaa700 1 -- 172.21.15.79:0/2218796832 shutdown_connections 2023-06-23T18:37:08.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.906+0000 7f3c9beaa700 1 -- 172.21.15.79:0/2218796832 wait complete. 2023-06-23T18:37:08.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.907+0000 7f3c9beaa700 1 Processor -- start 2023-06-23T18:37:08.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.907+0000 7f3c9beaa700 1 -- start start 2023-06-23T18:37:08.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9beaa700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c80004120 0x7f3c940b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:08.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9beaa700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c94152440 0x7f3c9414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:08.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9beaa700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9415bd70 0x7f3c9415cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:08.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9beaa700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9415a7f0 con 0x7f3c9415bd70 2023-06-23T18:37:08.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9beaa700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c941528b0 con 0x7f3c80004120 2023-06-23T18:37:08.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9beaa700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c94152b90 con 0x7f3c94152440 2023-06-23T18:37:08.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9a447700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9415bd70 0x7f3c9415cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:08.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9a447700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9415bd70 0x7f3c9415cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41780/0 (socket says 172.21.15.79:41780) 2023-06-23T18:37:08.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9a447700 1 -- 172.21.15.79:0/1850836795 learned_addr learned my addr 172.21.15.79:0/1850836795 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:08.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c99c46700 1 --2- 172.21.15.79:0/1850836795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c80004120 0x7f3c940b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:08.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c99445700 1 --2- 172.21.15.79:0/1850836795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c94152440 0x7f3c9414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:08.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9a447700 1 -- 172.21.15.79:0/1850836795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c94152440 msgr2=0x7f3c9414cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:08.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9a447700 1 --2- 172.21.15.79:0/1850836795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c94152440 0x7f3c9414cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:08.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9a447700 1 -- 172.21.15.79:0/1850836795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c80004120 msgr2=0x7f3c940b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:08.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9a447700 1 --2- 172.21.15.79:0/1850836795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c80004120 0x7f3c940b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:08.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9a447700 1 -- 172.21.15.79:0/1850836795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9415aa60 con 0x7f3c9415bd70 2023-06-23T18:37:08.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9a447700 1 --2- 172.21.15.79:0/1850836795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9415bd70 0x7f3c9415cf40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3c940b59b0 tx=0x7f3c8c00a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:08.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c8a7fc700 1 -- 172.21.15.79:0/1850836795 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c00ac80 con 0x7f3c9415bd70 2023-06-23T18:37:08.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c8a7fc700 1 -- 172.21.15.79:0/1850836795 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8c00ade0 con 0x7f3c9415bd70 2023-06-23T18:37:08.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c9beaa700 1 -- 172.21.15.79:0/1850836795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c940b2be0 con 0x7f3c9415bd70 2023-06-23T18:37:08.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.908+0000 7f3c8a7fc700 1 -- 172.21.15.79:0/1850836795 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c013630 con 0x7f3c9415bd70 2023-06-23T18:37:08.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.909+0000 7f3c9beaa700 1 -- 172.21.15.79:0/1850836795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9414d190 con 0x7f3c9415bd70 2023-06-23T18:37:08.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.909+0000 7f3c9beaa700 1 -- 172.21.15.79:0/1850836795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c940672c0 con 0x7f3c9415bd70 2023-06-23T18:37:08.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.910+0000 7f3c8a7fc700 1 -- 172.21.15.79:0/1850836795 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3c8c013790 con 0x7f3c9415bd70 2023-06-23T18:37:08.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.910+0000 7f3c8a7fc700 1 --2- 172.21.15.79:0/1850836795 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3c7806ace0 0x7f3c7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:08.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.910+0000 7f3c8a7fc700 1 -- 172.21.15.79:0/1850836795 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3c8c089f80 con 0x7f3c9415bd70 2023-06-23T18:37:08.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.910+0000 7f3c99c46700 1 --2- 172.21.15.79:0/1850836795 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3c7806ace0 0x7f3c7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:08.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.911+0000 7f3c99c46700 1 --2- 172.21.15.79:0/1850836795 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3c7806ace0 0x7f3c7806d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009b80 tx=0x7f3c90009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:08.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:08.914+0000 7f3c8a7fc700 1 -- 172.21.15.79:0/1850836795 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3c8c0550c0 con 0x7f3c9415bd70 2023-06-23T18:37:09.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.099+0000 7f3c9beaa700 1 -- 172.21.15.79:0/1850836795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3c940b1be0 con 0x7f3c9415bd70 2023-06-23T18:37:09.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.099+0000 7f3c8a7fc700 1 -- 172.21.15.79:0/1850836795 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3c8c017020 con 0x7f3c9415bd70 2023-06-23T18:37:09.105 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:37:09.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.101+0000 7f3c9beaa700 1 -- 172.21.15.79:0/1850836795 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3c7806ace0 msgr2=0x7f3c7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.101+0000 7f3c9beaa700 1 --2- 172.21.15.79:0/1850836795 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3c7806ace0 0x7f3c7806d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009b80 tx=0x7f3c90009210 comp rx=0 tx=0).stop 2023-06-23T18:37:09.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.101+0000 7f3c9beaa700 1 -- 172.21.15.79:0/1850836795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9415bd70 msgr2=0x7f3c9415cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.101+0000 7f3c9beaa700 1 --2- 172.21.15.79:0/1850836795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9415bd70 0x7f3c9415cf40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3c940b59b0 tx=0x7f3c8c00a5b0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.101+0000 7f3c9beaa700 1 -- 172.21.15.79:0/1850836795 shutdown_connections 2023-06-23T18:37:09.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.101+0000 7f3c9beaa700 1 --2- 172.21.15.79:0/1850836795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9415bd70 0x7f3c9415cf40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.101+0000 7f3c9beaa700 1 --2- 172.21.15.79:0/1850836795 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3c7806ace0 0x7f3c7806d190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.101+0000 7f3c9beaa700 1 --2- 172.21.15.79:0/1850836795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c80004120 0x7f3c940b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.101+0000 7f3c9beaa700 1 --2- 172.21.15.79:0/1850836795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c94152440 0x7f3c9414cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.101+0000 7f3c9beaa700 1 -- 172.21.15.79:0/1850836795 >> 172.21.15.79:0/1850836795 conn(0x7f3c9406afa0 msgr2=0x7f3c940cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:09.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.101+0000 7f3c9beaa700 1 -- 172.21.15.79:0/1850836795 shutdown_connections 2023-06-23T18:37:09.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.101+0000 7f3c9beaa700 1 -- 172.21.15.79:0/1850836795 wait complete. 2023-06-23T18:37:09.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:37:09.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.236+0000 7f887f47f700 1 Processor -- start 2023-06-23T18:37:09.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.237+0000 7f887f47f700 1 -- start start 2023-06-23T18:37:09.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.237+0000 7f887f47f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f887806a5a0 0x7f88781608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.237+0000 7f887f47f700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f88780cd440 0x7f88780b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.237+0000 7f887f47f700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88780cdc90 con 0x7f88781569c0 2023-06-23T18:37:09.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.237+0000 7f887f47f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88780cf570 con 0x7f887806a5a0 2023-06-23T18:37:09.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.237+0000 7f887f47f700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887815c8b0 con 0x7f88780cd440 2023-06-23T18:37:09.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.237+0000 7f887ca1a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f887806a5a0 0x7f88781608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.238+0000 7f887ca1a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f887806a5a0 0x7f88781608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41786/0 (socket says 172.21.15.79:41786) 2023-06-23T18:37:09.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.238+0000 7f887ca1a700 1 -- 172.21.15.79:0/1799619091 learned_addr learned my addr 172.21.15.79:0/1799619091 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:09.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.238+0000 7f887da1c700 1 --2- 172.21.15.79:0/1799619091 >> v2:172.21.15.155:3300/0 conn(0x7f88780cd440 0x7f88780b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.238+0000 7f887ca1a700 1 -- 172.21.15.79:0/1799619091 >> v2:172.21.15.155:3300/0 conn(0x7f88780cd440 msgr2=0x7f88780b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:09.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.238+0000 7f887ca1a700 1 --2- 172.21.15.79:0/1799619091 >> v2:172.21.15.155:3300/0 conn(0x7f88780cd440 0x7f88780b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.238+0000 7f887ca1a700 1 -- 172.21.15.79:0/1799619091 >> v1:172.21.15.155:6789/0 conn(0x7f88781569c0 legacy=0x7f8878069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:09.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.238+0000 7f887ca1a700 1 -- 172.21.15.79:0/1799619091 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887815d0f0 con 0x7f887806a5a0 2023-06-23T18:37:09.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.238+0000 7f887ca1a700 1 --2- 172.21.15.79:0/1799619091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f887806a5a0 0x7f88781608b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f887806e8f0 tx=0x7f8874009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9391cb73c3438e72 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:09.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.238+0000 7f886ffff700 1 -- 172.21.15.79:0/1799619091 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874004230 con 0x7f887806a5a0 2023-06-23T18:37:09.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.238+0000 7f886ffff700 1 -- 172.21.15.79:0/1799619091 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8874004390 con 0x7f887806a5a0 2023-06-23T18:37:09.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.238+0000 7f886ffff700 1 -- 172.21.15.79:0/1799619091 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740036a0 con 0x7f887806a5a0 2023-06-23T18:37:09.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.239+0000 7f887f47f700 1 -- 172.21.15.79:0/1799619091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f887806a5a0 msgr2=0x7f88781608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.239+0000 7f887f47f700 1 --2- 172.21.15.79:0/1799619091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f887806a5a0 0x7f88781608b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f887806e8f0 tx=0x7f8874009bd0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.239+0000 7f887f47f700 1 -- 172.21.15.79:0/1799619091 shutdown_connections 2023-06-23T18:37:09.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.239+0000 7f887f47f700 1 --2- 172.21.15.79:0/1799619091 >> v2:172.21.15.155:3300/0 conn(0x7f88780cd440 0x7f88780b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.239+0000 7f887f47f700 1 --2- 172.21.15.79:0/1799619091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f887806a5a0 0x7f88781608b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.239+0000 7f887f47f700 1 -- 172.21.15.79:0/1799619091 >> 172.21.15.79:0/1799619091 conn(0x7f887806afa0 msgr2=0x7f887806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:09.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.239+0000 7f887f47f700 1 -- 172.21.15.79:0/1799619091 shutdown_connections 2023-06-23T18:37:09.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.239+0000 7f887f47f700 1 -- 172.21.15.79:0/1799619091 wait complete. 2023-06-23T18:37:09.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.240+0000 7f887f47f700 1 Processor -- start 2023-06-23T18:37:09.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.240+0000 7f887f47f700 1 -- start start 2023-06-23T18:37:09.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887f47f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f887806a5a0 0x7f88780b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887f47f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88780cd440 0x7f88781f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887f47f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88781569c0 0x7f88780b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887f47f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88780b33d0 con 0x7f887806a5a0 2023-06-23T18:37:09.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887f47f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88781f1ce0 con 0x7f88781569c0 2023-06-23T18:37:09.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887f47f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88780b2be0 con 0x7f88780cd440 2023-06-23T18:37:09.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887d21b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f887806a5a0 0x7f88780b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887d21b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f887806a5a0 0x7f88780b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41790/0 (socket says 172.21.15.79:41790) 2023-06-23T18:37:09.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887d21b700 1 -- 172.21.15.79:0/3328075429 learned_addr learned my addr 172.21.15.79:0/3328075429 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:09.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887d21b700 1 -- 172.21.15.79:0/3328075429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88780cd440 msgr2=0x7f88781f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887ca1a700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88780cd440 0x7f88781f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887da1c700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88781569c0 0x7f88780b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887d21b700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88780cd440 0x7f88781f26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887d21b700 1 -- 172.21.15.79:0/3328075429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88781569c0 msgr2=0x7f88780b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887d21b700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88781569c0 0x7f88780b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887d21b700 1 -- 172.21.15.79:0/3328075429 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8874009550 con 0x7f887806a5a0 2023-06-23T18:37:09.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887da1c700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88781569c0 0x7f88780b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:09.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887d21b700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f887806a5a0 0x7f88780b26a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f886800b4a0 tx=0x7f886800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:09.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f886dffb700 1 -- 172.21.15.79:0/3328075429 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868011830 con 0x7f887806a5a0 2023-06-23T18:37:09.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f886dffb700 1 -- 172.21.15.79:0/3328075429 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8868011990 con 0x7f887806a5a0 2023-06-23T18:37:09.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f887f47f700 1 -- 172.21.15.79:0/3328075429 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88780b2e10 con 0x7f887806a5a0 2023-06-23T18:37:09.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.241+0000 7f886dffb700 1 -- 172.21.15.79:0/3328075429 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868011d40 con 0x7f887806a5a0 2023-06-23T18:37:09.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.242+0000 7f887f47f700 1 -- 172.21.15.79:0/3328075429 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88781f1ef0 con 0x7f887806a5a0 2023-06-23T18:37:09.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.243+0000 7f887f47f700 1 -- 172.21.15.79:0/3328075429 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88780672c0 con 0x7f887806a5a0 2023-06-23T18:37:09.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.243+0000 7f886dffb700 1 -- 172.21.15.79:0/3328075429 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f886801a400 con 0x7f887806a5a0 2023-06-23T18:37:09.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.243+0000 7f886dffb700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f885c06ace0 0x7f885c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.243+0000 7f886dffb700 1 -- 172.21.15.79:0/3328075429 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f88680891c0 con 0x7f887806a5a0 2023-06-23T18:37:09.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.243+0000 7f887ca1a700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f885c06ace0 0x7f885c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.244+0000 7f887ca1a700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f885c06ace0 0x7f885c06d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f887806a990 tx=0x7f8874008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:09.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.248+0000 7f886dffb700 1 -- 172.21.15.79:0/3328075429 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8868054270 con 0x7f887806a5a0 2023-06-23T18:37:09.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.433+0000 7f887f47f700 1 -- 172.21.15.79:0/3328075429 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88780cc2f0 con 0x7f887806a5a0 2023-06-23T18:37:09.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.433+0000 7f886dffb700 1 -- 172.21.15.79:0/3328075429 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8868057890 con 0x7f887806a5a0 2023-06-23T18:37:09.439 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:37:09.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.436+0000 7f887f47f700 1 -- 172.21.15.79:0/3328075429 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f885c06ace0 msgr2=0x7f885c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.436+0000 7f887f47f700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f885c06ace0 0x7f885c06d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f887806a990 tx=0x7f8874008030 comp rx=0 tx=0).stop 2023-06-23T18:37:09.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.436+0000 7f887f47f700 1 -- 172.21.15.79:0/3328075429 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f887806a5a0 msgr2=0x7f88780b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.436+0000 7f887f47f700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f887806a5a0 0x7f88780b26a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f886800b4a0 tx=0x7f886800b860 comp rx=0 tx=0).stop 2023-06-23T18:37:09.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.436+0000 7f887f47f700 1 -- 172.21.15.79:0/3328075429 shutdown_connections 2023-06-23T18:37:09.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.436+0000 7f887f47f700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f885c06ace0 0x7f885c06d190 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.436+0000 7f887f47f700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88781569c0 0x7f88780b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.436+0000 7f887f47f700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f887806a5a0 0x7f88780b26a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.436+0000 7f887f47f700 1 --2- 172.21.15.79:0/3328075429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88780cd440 0x7f88781f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.436+0000 7f887f47f700 1 -- 172.21.15.79:0/3328075429 >> 172.21.15.79:0/3328075429 conn(0x7f887806afa0 msgr2=0x7f88780cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:09.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.436+0000 7f887f47f700 1 -- 172.21.15.79:0/3328075429 shutdown_connections 2023-06-23T18:37:09.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.436+0000 7f887f47f700 1 -- 172.21.15.79:0/3328075429 wait complete. 2023-06-23T18:37:09.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:37:09.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.571+0000 7f72fc74d700 1 Processor -- start 2023-06-23T18:37:09.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.571+0000 7f72fc74d700 1 -- start start 2023-06-23T18:37:09.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72fc74d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72f41569c0 0x7f72f41608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72fc74d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f40cdc90 con 0x7f72f40cd440 2023-06-23T18:37:09.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72fc74d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f415d0f0 con 0x7f72f406a5a0 2023-06-23T18:37:09.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72fc74d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f40cf570 con 0x7f72f41569c0 2023-06-23T18:37:09.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72f9ce8700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f72f40cd440 0x7f72f4069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44840/0 (socket says 172.21.15.79:44840) 2023-06-23T18:37:09.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72f9ce8700 1 -- 172.21.15.79:0/1173638695 learned_addr learned my addr 172.21.15.79:0/1173638695 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:09.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72fa4e9700 1 --2- 172.21.15.79:0/1173638695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72f41569c0 0x7f72f41608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72f94e7700 1 -- 172.21.15.79:0/1173638695 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782638082 0 0) 0x7f72e4001110 con 0x7f72f40cd440 2023-06-23T18:37:09.578 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72f94e7700 1 -- 172.21.15.79:0/1173638695 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72e00035b0 con 0x7f72f40cd440 2023-06-23T18:37:09.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72f94e7700 1 -- 172.21.15.79:0/1173638695 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352842799 0 0) 0x7f72f415d0f0 con 0x7f72f406a5a0 2023-06-23T18:37:09.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72f94e7700 1 -- 172.21.15.79:0/1173638695 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72e4001110 con 0x7f72f406a5a0 2023-06-23T18:37:09.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72fa4e9700 1 -- 172.21.15.79:0/1173638695 >> v1:172.21.15.155:6789/0 conn(0x7f72f406a5a0 legacy=0x7f72f40b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72f94e7700 1 -- 172.21.15.79:0/1173638695 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2610433021 0 0) 0x7f72e00035b0 con 0x7f72f40cd440 2023-06-23T18:37:09.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72fa4e9700 1 -- 172.21.15.79:0/1173638695 >> v1:172.21.15.79:6789/0 conn(0x7f72f40cd440 legacy=0x7f72f4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.572+0000 7f72fa4e9700 1 -- 172.21.15.79:0/1173638695 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f4156d90 con 0x7f72f41569c0 2023-06-23T18:37:09.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.573+0000 7f72fa4e9700 1 --2- 172.21.15.79:0/1173638695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72f41569c0 0x7f72f41608b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009870 tx=0x7f72f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e76a56a74f1840c8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:09.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.573+0000 7f72f94e7700 1 -- 172.21.15.79:0/1173638695 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4001800 con 0x7f72f41569c0 2023-06-23T18:37:09.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.573+0000 7f72f94e7700 1 -- 172.21.15.79:0/1173638695 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f0004030 con 0x7f72f41569c0 2023-06-23T18:37:09.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.573+0000 7f72f94e7700 1 -- 172.21.15.79:0/1173638695 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f00043c0 con 0x7f72f41569c0 2023-06-23T18:37:09.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.573+0000 7f72fc74d700 1 -- 172.21.15.79:0/1173638695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72f41569c0 msgr2=0x7f72f41608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.573+0000 7f72fc74d700 1 --2- 172.21.15.79:0/1173638695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72f41569c0 0x7f72f41608b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009870 tx=0x7f72f0009b80 comp rx=0 tx=0).stop 2023-06-23T18:37:09.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.573+0000 7f72fc74d700 1 -- 172.21.15.79:0/1173638695 shutdown_connections 2023-06-23T18:37:09.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.573+0000 7f72fc74d700 1 --2- 172.21.15.79:0/1173638695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72f41569c0 0x7f72f41608b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.573+0000 7f72fc74d700 1 -- 172.21.15.79:0/1173638695 >> 172.21.15.79:0/1173638695 conn(0x7f72f406afa0 msgr2=0x7f72f406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:09.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.573+0000 7f72fc74d700 1 -- 172.21.15.79:0/1173638695 shutdown_connections 2023-06-23T18:37:09.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.573+0000 7f72fc74d700 1 -- 172.21.15.79:0/1173638695 wait complete. 2023-06-23T18:37:09.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.575+0000 7f72fc74d700 1 Processor -- start 2023-06-23T18:37:09.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.575+0000 7f72fc74d700 1 -- start start 2023-06-23T18:37:09.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.575+0000 7f72fc74d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72f406a5a0 0x7f72f414f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.575+0000 7f72fc74d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72f40cd440 0x7f72f40c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.575+0000 7f72fc74d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f41569c0 0x7f72f40c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.575+0000 7f72fc74d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f414fde0 con 0x7f72f40cd440 2023-06-23T18:37:09.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.575+0000 7f72fc74d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f40c4590 con 0x7f72f41569c0 2023-06-23T18:37:09.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.575+0000 7f72fc74d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f40c4870 con 0x7f72f406a5a0 2023-06-23T18:37:09.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.575+0000 7f72f9ce8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72f40cd440 0x7f72f40c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.575+0000 7f72f9ce8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72f40cd440 0x7f72f40c5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41806/0 (socket says 172.21.15.79:41806) 2023-06-23T18:37:09.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.575+0000 7f72f9ce8700 1 -- 172.21.15.79:0/19794155 learned_addr learned my addr 172.21.15.79:0/19794155 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:09.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.576+0000 7f72facea700 1 --2- 172.21.15.79:0/19794155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f41569c0 0x7f72f40c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.576+0000 7f72fa4e9700 1 --2- 172.21.15.79:0/19794155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72f406a5a0 0x7f72f414f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.576+0000 7f72f9ce8700 1 -- 172.21.15.79:0/19794155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72f406a5a0 msgr2=0x7f72f414f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.576+0000 7f72f9ce8700 1 --2- 172.21.15.79:0/19794155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72f406a5a0 0x7f72f414f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.576+0000 7f72f9ce8700 1 -- 172.21.15.79:0/19794155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f41569c0 msgr2=0x7f72f40c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.576+0000 7f72f9ce8700 1 --2- 172.21.15.79:0/19794155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f41569c0 0x7f72f40c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.576+0000 7f72f9ce8700 1 -- 172.21.15.79:0/19794155 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f00094a0 con 0x7f72f40cd440 2023-06-23T18:37:09.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.576+0000 7f72f9ce8700 1 --2- 172.21.15.79:0/19794155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72f40cd440 0x7f72f40c5630 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f72e400cac0 tx=0x7f72e400ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:09.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.576+0000 7f72eaffd700 1 -- 172.21.15.79:0/19794155 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4004a80 con 0x7f72f40cd440 2023-06-23T18:37:09.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.576+0000 7f72eaffd700 1 -- 172.21.15.79:0/19794155 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e4004be0 con 0x7f72f40cd440 2023-06-23T18:37:09.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.576+0000 7f72fc74d700 1 -- 172.21.15.79:0/19794155 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f40c4b50 con 0x7f72f40cd440 2023-06-23T18:37:09.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.576+0000 7f72eaffd700 1 -- 172.21.15.79:0/19794155 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e40070d0 con 0x7f72f40cd440 2023-06-23T18:37:09.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.576+0000 7f72fc74d700 1 -- 172.21.15.79:0/19794155 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f40c5110 con 0x7f72f40cd440 2023-06-23T18:37:09.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.577+0000 7f72fc74d700 1 -- 172.21.15.79:0/19794155 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f40c5330 con 0x7f72f40cd440 2023-06-23T18:37:09.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.577+0000 7f72eaffd700 1 -- 172.21.15.79:0/19794155 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f72e4007230 con 0x7f72f40cd440 2023-06-23T18:37:09.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.578+0000 7f72eaffd700 1 --2- 172.21.15.79:0/19794155 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f72d806ace0 0x7f72d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.578+0000 7f72fa4e9700 1 --2- 172.21.15.79:0/19794155 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f72d806ace0 0x7f72d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.578+0000 7f72eaffd700 1 -- 172.21.15.79:0/19794155 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f72e40886f0 con 0x7f72f40cd440 2023-06-23T18:37:09.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.578+0000 7f72fa4e9700 1 --2- 172.21.15.79:0/19794155 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f72d806ace0 0x7f72d806d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f72f00097c0 tx=0x7f72f0013040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:09.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.582+0000 7f72eaffd700 1 -- 172.21.15.79:0/19794155 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f72e4053690 con 0x7f72f40cd440 2023-06-23T18:37:09.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.768+0000 7f72fc74d700 1 -- 172.21.15.79:0/19794155 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f72f40cdc90 con 0x7f72f40cd440 2023-06-23T18:37:09.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.768+0000 7f72eaffd700 1 -- 172.21.15.79:0/19794155 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f72f40cdc90 con 0x7f72f40cd440 2023-06-23T18:37:09.774 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:37:09.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.770+0000 7f72fc74d700 1 -- 172.21.15.79:0/19794155 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f72d806ace0 msgr2=0x7f72d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.770+0000 7f72fc74d700 1 --2- 172.21.15.79:0/19794155 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f72d806ace0 0x7f72d806d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f72f00097c0 tx=0x7f72f0013040 comp rx=0 tx=0).stop 2023-06-23T18:37:09.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.770+0000 7f72fc74d700 1 -- 172.21.15.79:0/19794155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72f40cd440 msgr2=0x7f72f40c5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.770+0000 7f72fc74d700 1 --2- 172.21.15.79:0/19794155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72f40cd440 0x7f72f40c5630 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f72e400cac0 tx=0x7f72e400ce80 comp rx=0 tx=0).stop 2023-06-23T18:37:09.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.770+0000 7f72fc74d700 1 -- 172.21.15.79:0/19794155 shutdown_connections 2023-06-23T18:37:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.770+0000 7f72fc74d700 1 --2- 172.21.15.79:0/19794155 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f72d806ace0 0x7f72d806d190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.770+0000 7f72fc74d700 1 --2- 172.21.15.79:0/19794155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f41569c0 0x7f72f40c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.770+0000 7f72fc74d700 1 --2- 172.21.15.79:0/19794155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72f406a5a0 0x7f72f414f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.770+0000 7f72fc74d700 1 --2- 172.21.15.79:0/19794155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72f40cd440 0x7f72f40c5630 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.770+0000 7f72fc74d700 1 -- 172.21.15.79:0/19794155 >> 172.21.15.79:0/19794155 conn(0x7f72f406afa0 msgr2=0x7f72f40ca640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.770+0000 7f72fc74d700 1 -- 172.21.15.79:0/19794155 shutdown_connections 2023-06-23T18:37:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.771+0000 7f72fc74d700 1 -- 172.21.15.79:0/19794155 wait complete. 2023-06-23T18:37:09.787 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:37:09.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.905+0000 7fe25d499700 1 Processor -- start 2023-06-23T18:37:09.911 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.905+0000 7fe25d499700 1 -- start start 2023-06-23T18:37:09.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.906+0000 7fe25d499700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe2580cd440 0x7fe2581608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.906+0000 7fe25d499700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2580cdc90 con 0x7fe2581569c0 2023-06-23T18:37:09.913 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.906+0000 7fe25d499700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe25815d0f0 con 0x7fe25806a5a0 2023-06-23T18:37:09.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.906+0000 7fe25d499700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2580cf570 con 0x7fe2580cd440 2023-06-23T18:37:09.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.906+0000 7fe256ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe2581569c0 0x7fe258069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44854/0 (socket says 172.21.15.79:44854) 2023-06-23T18:37:09.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.906+0000 7fe256ffd700 1 -- 172.21.15.79:0/4025322894 learned_addr learned my addr 172.21.15.79:0/4025322894 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:09.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 -- 172.21.15.79:0/4025322894 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115789681 0 0) 0x7fe24c001110 con 0x7fe2581569c0 2023-06-23T18:37:09.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe2577fe700 1 --2- 172.21.15.79:0/4025322894 >> v2:172.21.15.155:3300/0 conn(0x7fe2580cd440 0x7fe2581608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 -- 172.21.15.79:0/4025322894 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2400035b0 con 0x7fe2581569c0 2023-06-23T18:37:09.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 -- 172.21.15.79:0/4025322894 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1336159748 0 0) 0x7fe2400035b0 con 0x7fe2581569c0 2023-06-23T18:37:09.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 -- 172.21.15.79:0/4025322894 >> v2:172.21.15.155:3300/0 conn(0x7fe2580cd440 msgr2=0x7fe2581608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:09.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 --2- 172.21.15.79:0/4025322894 >> v2:172.21.15.155:3300/0 conn(0x7fe2580cd440 0x7fe2581608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 -- 172.21.15.79:0/4025322894 >> v1:172.21.15.155:6789/0 conn(0x7fe25806a5a0 legacy=0x7fe2580b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 -- 172.21.15.79:0/4025322894 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe258156d90 con 0x7fe2581569c0 2023-06-23T18:37:09.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 -- 172.21.15.79:0/4025322894 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fe24c001800 con 0x7fe2581569c0 2023-06-23T18:37:09.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 -- 172.21.15.79:0/4025322894 >> v1:172.21.15.79:6789/0 conn(0x7fe2581569c0 legacy=0x7fe258069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 --2- 172.21.15.79:0/4025322894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe240003a00 0x7fe240003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 --2- 172.21.15.79:0/4025322894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe240004850 0x7fe240004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 --2- 172.21.15.79:0/4025322894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe240005200 0x7fe24000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 -- 172.21.15.79:0/4025322894 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24000c050 con 0x7fe240003a00 2023-06-23T18:37:09.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 -- 172.21.15.79:0/4025322894 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24000c1c0 con 0x7fe240004850 2023-06-23T18:37:09.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe255ffb700 1 -- 172.21.15.79:0/4025322894 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24000c380 con 0x7fe240005200 2023-06-23T18:37:09.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe256ffd700 1 --2- 172.21.15.79:0/4025322894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe240003a00 0x7fe240003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe2567fc700 1 --2- 172.21.15.79:0/4025322894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe240004850 0x7fe240004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe2577fe700 1 --2- 172.21.15.79:0/4025322894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe240005200 0x7fe24000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe256ffd700 1 -- 172.21.15.79:0/4025322894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe240005200 msgr2=0x7fe24000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:09.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe256ffd700 1 --2- 172.21.15.79:0/4025322894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe240005200 0x7fe24000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe256ffd700 1 -- 172.21.15.79:0/4025322894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe240004850 msgr2=0x7fe240004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:09.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe256ffd700 1 --2- 172.21.15.79:0/4025322894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe240004850 0x7fe240004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe256ffd700 1 -- 172.21.15.79:0/4025322894 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2400035b0 con 0x7fe240003a00 2023-06-23T18:37:09.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe2577fe700 1 -- 172.21.15.79:0/4025322894 reap_dead start 2023-06-23T18:37:09.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.907+0000 7fe256ffd700 1 --2- 172.21.15.79:0/4025322894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe240003a00 0x7fe240003e10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00c240 tx=0x7fe240005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b2e377701fa4cb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:09.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.908+0000 7fe255ffb700 1 -- 172.21.15.79:0/4025322894 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c007db0 con 0x7fe240003a00 2023-06-23T18:37:09.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.908+0000 7fe255ffb700 1 -- 172.21.15.79:0/4025322894 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24c00cd60 con 0x7fe240003a00 2023-06-23T18:37:09.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.908+0000 7fe255ffb700 1 -- 172.21.15.79:0/4025322894 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c005b80 con 0x7fe240003a00 2023-06-23T18:37:09.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.908+0000 7fe25d499700 1 -- 172.21.15.79:0/4025322894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe240003a00 msgr2=0x7fe240003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.908+0000 7fe25d499700 1 --2- 172.21.15.79:0/4025322894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe240003a00 0x7fe240003e10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00c240 tx=0x7fe240005880 comp rx=0 tx=0).stop 2023-06-23T18:37:09.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.908+0000 7fe25d499700 1 -- 172.21.15.79:0/4025322894 shutdown_connections 2023-06-23T18:37:09.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.908+0000 7fe25d499700 1 --2- 172.21.15.79:0/4025322894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe240003a00 0x7fe240003e10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.908+0000 7fe25d499700 1 -- 172.21.15.79:0/4025322894 >> 172.21.15.79:0/4025322894 conn(0x7fe25806afa0 msgr2=0x7fe25806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:09.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.908+0000 7fe25d499700 1 -- 172.21.15.79:0/4025322894 shutdown_connections 2023-06-23T18:37:09.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.908+0000 7fe25d499700 1 -- 172.21.15.79:0/4025322894 wait complete. 2023-06-23T18:37:09.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.910+0000 7fe25d499700 1 Processor -- start 2023-06-23T18:37:09.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.910+0000 7fe25d499700 1 -- start start 2023-06-23T18:37:09.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe25d499700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe240003a00 0x7fe25814ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe25d499700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2580b3300 0x7fe2580c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe25d499700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2580c42f0 0x7fe2580c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe25d499700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2580c4800 con 0x7fe240003a00 2023-06-23T18:37:09.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe25d499700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2580c7480 con 0x7fe2580b3300 2023-06-23T18:37:09.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe25d499700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25814b390 con 0x7fe2580c42f0 2023-06-23T18:37:09.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe256ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe240003a00 0x7fe25814ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe256ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe240003a00 0x7fe25814ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41824/0 (socket says 172.21.15.79:41824) 2023-06-23T18:37:09.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe256ffd700 1 -- 172.21.15.79:0/4280144541 learned_addr learned my addr 172.21.15.79:0/4280144541 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:09.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe2567fc700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2580b3300 0x7fe2580c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe2577fe700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2580c42f0 0x7fe2580c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe256ffd700 1 -- 172.21.15.79:0/4280144541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2580c42f0 msgr2=0x7fe2580c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe256ffd700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2580c42f0 0x7fe2580c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe256ffd700 1 -- 172.21.15.79:0/4280144541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2580b3300 msgr2=0x7fe2580c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:09.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe256ffd700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2580b3300 0x7fe2580c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:09.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe256ffd700 1 -- 172.21.15.79:0/4280144541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2580c6ae0 con 0x7fe240003a00 2023-06-23T18:37:09.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe2577fe700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2580c42f0 0x7fe2580c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:09.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe256ffd700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe240003a00 0x7fe25814ae50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe24c000c00 tx=0x7fe24c001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:09.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.911+0000 7fe23f7fe700 1 -- 172.21.15.79:0/4280144541 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c0017c0 con 0x7fe240003a00 2023-06-23T18:37:09.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.912+0000 7fe23f7fe700 1 -- 172.21.15.79:0/4280144541 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24c00cbe0 con 0x7fe240003a00 2023-06-23T18:37:09.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.912+0000 7fe25d499700 1 -- 172.21.15.79:0/4280144541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2580c6dc0 con 0x7fe240003a00 2023-06-23T18:37:09.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.912+0000 7fe23f7fe700 1 -- 172.21.15.79:0/4280144541 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c007330 con 0x7fe240003a00 2023-06-23T18:37:09.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.912+0000 7fe25d499700 1 -- 172.21.15.79:0/4280144541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2580c50e0 con 0x7fe240003a00 2023-06-23T18:37:09.931 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.913+0000 7fe25d499700 1 -- 172.21.15.79:0/4280144541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2580c75c0 con 0x7fe240003a00 2023-06-23T18:37:09.931 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.913+0000 7fe23f7fe700 1 -- 172.21.15.79:0/4280144541 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe24c007490 con 0x7fe240003a00 2023-06-23T18:37:09.931 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.913+0000 7fe23f7fe700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe24406ace0 0x7fe24406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:09.931 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.913+0000 7fe23f7fe700 1 -- 172.21.15.79:0/4280144541 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe24c08a4e0 con 0x7fe240003a00 2023-06-23T18:37:09.931 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.913+0000 7fe2567fc700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe24406ace0 0x7fe24406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:09.932 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.914+0000 7fe2567fc700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe24406ace0 0x7fe24406d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe248000c00 tx=0x7fe248009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:09.932 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:09.918+0000 7fe23f7fe700 1 -- 172.21.15.79:0/4280144541 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe2580c75c0 con 0x7fe240003a00 2023-06-23T18:37:10.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.105+0000 7fe25d499700 1 -- 172.21.15.79:0/4280144541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2580cdc90 con 0x7fe240003a00 2023-06-23T18:37:10.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.105+0000 7fe23f7fe700 1 -- 172.21.15.79:0/4280144541 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe2580cdc90 con 0x7fe240003a00 2023-06-23T18:37:10.110 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:37:10.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.107+0000 7fe25d499700 1 -- 172.21.15.79:0/4280144541 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe24406ace0 msgr2=0x7fe24406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.107+0000 7fe25d499700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe24406ace0 0x7fe24406d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe248000c00 tx=0x7fe248009040 comp rx=0 tx=0).stop 2023-06-23T18:37:10.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.107+0000 7fe25d499700 1 -- 172.21.15.79:0/4280144541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe240003a00 msgr2=0x7fe25814ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.107+0000 7fe25d499700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe240003a00 0x7fe25814ae50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe24c000c00 tx=0x7fe24c001b90 comp rx=0 tx=0).stop 2023-06-23T18:37:10.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.107+0000 7fe25d499700 1 -- 172.21.15.79:0/4280144541 shutdown_connections 2023-06-23T18:37:10.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.107+0000 7fe25d499700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2580c42f0 0x7fe2580c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.107+0000 7fe25d499700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe240003a00 0x7fe25814ae50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.107+0000 7fe25d499700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe24406ace0 0x7fe24406d190 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.107+0000 7fe25d499700 1 --2- 172.21.15.79:0/4280144541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2580b3300 0x7fe2580c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.107+0000 7fe25d499700 1 -- 172.21.15.79:0/4280144541 >> 172.21.15.79:0/4280144541 conn(0x7fe25806afa0 msgr2=0x7fe25806de90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:10.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.107+0000 7fe25d499700 1 -- 172.21.15.79:0/4280144541 shutdown_connections 2023-06-23T18:37:10.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.107+0000 7fe25d499700 1 -- 172.21.15.79:0/4280144541 wait complete. 2023-06-23T18:37:10.124 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:37:10.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.240+0000 7f7e6df9d700 1 Processor -- start 2023-06-23T18:37:10.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.241+0000 7f7e6df9d700 1 -- start start 2023-06-23T18:37:10.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.241+0000 7f7e6df9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7e680b83c0 0x7f7e680cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:10.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.241+0000 7f7e6df9d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e680d0a80 con 0x7f7e680cfc60 2023-06-23T18:37:10.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.241+0000 7f7e6df9d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e680b8790 con 0x7f7e680d3710 2023-06-23T18:37:10.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.241+0000 7f7e6df9d700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e680b4e70 con 0x7f7e680b83c0 2023-06-23T18:37:10.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.241+0000 7f7e677fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7e680b83c0 0x7f7e680cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:10.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.241+0000 7f7e67fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f7e680cfc60 0x7f7e680d0f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44878/0 (socket says 172.21.15.79:44878) 2023-06-23T18:37:10.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.241+0000 7f7e67fff700 1 -- 172.21.15.79:0/2828789236 learned_addr learned my addr 172.21.15.79:0/2828789236 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:10.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.242+0000 7f7e677fe700 1 -- 172.21.15.79:0/2828789236 >> v1:172.21.15.155:6789/0 conn(0x7f7e680d3710 legacy=0x7f7e680cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.242+0000 7f7e677fe700 1 -- 172.21.15.79:0/2828789236 >> v1:172.21.15.79:6789/0 conn(0x7f7e680cfc60 legacy=0x7f7e680d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.242+0000 7f7e677fe700 1 -- 172.21.15.79:0/2828789236 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e680b5150 con 0x7f7e680b83c0 2023-06-23T18:37:10.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.242+0000 7f7e667fc700 1 -- 172.21.15.79:0/2828789236 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988951274 0 0) 0x7f7e680d0a80 con 0x7f7e680cfc60 2023-06-23T18:37:10.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.242+0000 7f7e677fe700 1 --2- 172.21.15.79:0/2828789236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e680b83c0 0x7f7e680cd200 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009840 tx=0x7f7e5c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee2bdd434b16deb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:10.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.242+0000 7f7e667fc700 1 -- 172.21.15.79:0/2828789236 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c011830 con 0x7f7e680b83c0 2023-06-23T18:37:10.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.242+0000 7f7e667fc700 1 -- 172.21.15.79:0/2828789236 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c011990 con 0x7f7e680b83c0 2023-06-23T18:37:10.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.242+0000 7f7e667fc700 1 -- 172.21.15.79:0/2828789236 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c011d20 con 0x7f7e680b83c0 2023-06-23T18:37:10.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.243+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2828789236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e680b83c0 msgr2=0x7f7e680cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.243+0000 7f7e6df9d700 1 --2- 172.21.15.79:0/2828789236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e680b83c0 0x7f7e680cd200 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009840 tx=0x7f7e5c009b50 comp rx=0 tx=0).stop 2023-06-23T18:37:10.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.243+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2828789236 shutdown_connections 2023-06-23T18:37:10.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.243+0000 7f7e6df9d700 1 --2- 172.21.15.79:0/2828789236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e680b83c0 0x7f7e680cd200 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.243+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2828789236 >> 172.21.15.79:0/2828789236 conn(0x7f7e6806b350 msgr2=0x7f7e6806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:10.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.243+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2828789236 shutdown_connections 2023-06-23T18:37:10.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.243+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2828789236 wait complete. 2023-06-23T18:37:10.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.244+0000 7f7e6df9d700 1 Processor -- start 2023-06-23T18:37:10.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.244+0000 7f7e6df9d700 1 -- start start 2023-06-23T18:37:10.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e6df9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e680b83c0 0x7f7e68154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:10.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e6df9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e680cfc60 0x7f7e68159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:10.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e6df9d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e680d3710 0x7f7e68159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:10.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e6df9d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e681545c0 con 0x7f7e680b83c0 2023-06-23T18:37:10.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e6df9d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680cbdd0 con 0x7f7e680cfc60 2023-06-23T18:37:10.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e6df9d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680cc0b0 con 0x7f7e680d3710 2023-06-23T18:37:10.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e677fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e680b83c0 0x7f7e68154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:10.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e677fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e680b83c0 0x7f7e68154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41838/0 (socket says 172.21.15.79:41838) 2023-06-23T18:37:10.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e677fe700 1 -- 172.21.15.79:0/2195497134 learned_addr learned my addr 172.21.15.79:0/2195497134 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:10.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e66ffd700 1 --2- 172.21.15.79:0/2195497134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e680cfc60 0x7f7e68159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:10.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e677fe700 1 -- 172.21.15.79:0/2195497134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e680d3710 msgr2=0x7f7e68159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e677fe700 1 --2- 172.21.15.79:0/2195497134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e680d3710 0x7f7e68159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e677fe700 1 -- 172.21.15.79:0/2195497134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e680cfc60 msgr2=0x7f7e68159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e677fe700 1 --2- 172.21.15.79:0/2195497134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e680cfc60 0x7f7e68159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e677fe700 1 -- 172.21.15.79:0/2195497134 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c0094a0 con 0x7f7e680b83c0 2023-06-23T18:37:10.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e66ffd700 1 --2- 172.21.15.79:0/2195497134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e680cfc60 0x7f7e68159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:10.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e677fe700 1 --2- 172.21.15.79:0/2195497134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e680b83c0 0x7f7e68154080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0090b0 tx=0x7f7e5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:10.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e4ffff700 1 -- 172.21.15.79:0/2195497134 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0036a0 con 0x7f7e680b83c0 2023-06-23T18:37:10.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e4ffff700 1 -- 172.21.15.79:0/2195497134 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c01e070 con 0x7f7e680b83c0 2023-06-23T18:37:10.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2195497134 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e680cc390 con 0x7f7e680b83c0 2023-06-23T18:37:10.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e4ffff700 1 -- 172.21.15.79:0/2195497134 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c003a60 con 0x7f7e680b83c0 2023-06-23T18:37:10.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.245+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2195497134 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e680cb0e0 con 0x7f7e680b83c0 2023-06-23T18:37:10.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.246+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2195497134 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e680cf0b0 con 0x7f7e680b83c0 2023-06-23T18:37:10.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.247+0000 7f7e4ffff700 1 -- 172.21.15.79:0/2195497134 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e5c003bc0 con 0x7f7e680b83c0 2023-06-23T18:37:10.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.247+0000 7f7e4ffff700 1 --2- 172.21.15.79:0/2195497134 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e5406ace0 0x7f7e5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:10.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.247+0000 7f7e4ffff700 1 -- 172.21.15.79:0/2195497134 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7e5c08a620 con 0x7f7e680b83c0 2023-06-23T18:37:10.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.247+0000 7f7e66ffd700 1 --2- 172.21.15.79:0/2195497134 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e5406ace0 0x7f7e5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:10.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.248+0000 7f7e66ffd700 1 --2- 172.21.15.79:0/2195497134 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e5406ace0 0x7f7e5406d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7e681f5670 tx=0x7f7e58009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:10.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.252+0000 7f7e4ffff700 1 -- 172.21.15.79:0/2195497134 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7e5c055760 con 0x7f7e680b83c0 2023-06-23T18:37:10.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.452+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2195497134 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7e680cf910 con 0x7f7e680b83c0 2023-06-23T18:37:10.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.453+0000 7f7e4ffff700 1 -- 172.21.15.79:0/2195497134 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7e5c058d80 con 0x7f7e680b83c0 2023-06-23T18:37:10.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.455+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2195497134 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e5406ace0 msgr2=0x7f7e5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.455+0000 7f7e6df9d700 1 --2- 172.21.15.79:0/2195497134 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e5406ace0 0x7f7e5406d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7e681f5670 tx=0x7f7e58009040 comp rx=0 tx=0).stop 2023-06-23T18:37:10.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.455+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2195497134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e680b83c0 msgr2=0x7f7e68154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.455+0000 7f7e6df9d700 1 --2- 172.21.15.79:0/2195497134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e680b83c0 0x7f7e68154080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0090b0 tx=0x7f7e5c004750 comp rx=0 tx=0).stop 2023-06-23T18:37:10.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.455+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2195497134 shutdown_connections 2023-06-23T18:37:10.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.455+0000 7f7e6df9d700 1 --2- 172.21.15.79:0/2195497134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e680d3710 0x7f7e68159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.455+0000 7f7e6df9d700 1 --2- 172.21.15.79:0/2195497134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e680b83c0 0x7f7e68154080 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.455+0000 7f7e6df9d700 1 --2- 172.21.15.79:0/2195497134 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7e5406ace0 0x7f7e5406d190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.455+0000 7f7e6df9d700 1 --2- 172.21.15.79:0/2195497134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e680cfc60 0x7f7e68159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.455+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2195497134 >> 172.21.15.79:0/2195497134 conn(0x7f7e6806b350 msgr2=0x7f7e680b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:10.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.455+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2195497134 shutdown_connections 2023-06-23T18:37:10.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.455+0000 7f7e6df9d700 1 -- 172.21.15.79:0/2195497134 wait complete. 2023-06-23T18:37:10.462 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-23T18:37:10.472 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi079.front.sepia.ceph.com... 2023-06-23T18:37:10.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:10.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-06-23T18:37:10.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi079.stderr:ceph-fuse[116927]: fuse finished with error 0 and tester_r 0 2023-06-23T18:37:10.507 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:37:10.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:10.507 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-06-23T18:37:10.573 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:37:10.573 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:37:10.594 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:37:10.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.728+0000 7f68690fd700 1 Processor -- start 2023-06-23T18:37:10.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.728+0000 7f68690fd700 1 -- start start 2023-06-23T18:37:10.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f68690fd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68640cec00 0x7f68641602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:10.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f68690fd700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686415fe30 con 0x7f68640b17d0 2023-06-23T18:37:10.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f68690fd700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686406ce10 con 0x7f686415fa60 2023-06-23T18:37:10.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f68690fd700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68640b0a10 con 0x7f68640cec00 2023-06-23T18:37:10.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f686259c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68640cec00 0x7f68641602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:10.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f686359e700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f68640b17d0 0x7f6864160820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44892/0 (socket says 172.21.15.79:44892) 2023-06-23T18:37:10.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f686259c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68640cec00 0x7f68641602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41848/0 (socket says 172.21.15.79:41848) 2023-06-23T18:37:10.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f686359e700 1 -- 172.21.15.79:0/863396273 learned_addr learned my addr 172.21.15.79:0/863396273 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:10.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f686259c700 1 -- 172.21.15.79:0/863396273 >> v1:172.21.15.155:6789/0 conn(0x7f686415fa60 legacy=0x7f68640ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:10.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f686259c700 1 -- 172.21.15.79:0/863396273 >> v1:172.21.15.79:6789/0 conn(0x7f68640b17d0 legacy=0x7f6864160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f686259c700 1 -- 172.21.15.79:0/863396273 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68640b0cf0 con 0x7f68640cec00 2023-06-23T18:37:10.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f6861d9b700 1 -- 172.21.15.79:0/863396273 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695192458 0 0) 0x7f685c001490 con 0x7f68640b17d0 2023-06-23T18:37:10.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f686259c700 1 --2- 172.21.15.79:0/863396273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68640cec00 0x7f68641602e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6854009840 tx=0x7f6854009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe493124757c8c5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:10.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f6861d9b700 1 -- 172.21.15.79:0/863396273 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854004230 con 0x7f68640cec00 2023-06-23T18:37:10.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f6861d9b700 1 -- 172.21.15.79:0/863396273 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6854004390 con 0x7f68640cec00 2023-06-23T18:37:10.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.729+0000 7f6861d9b700 1 -- 172.21.15.79:0/863396273 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68540036a0 con 0x7f68640cec00 2023-06-23T18:37:10.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.730+0000 7f68690fd700 1 -- 172.21.15.79:0/863396273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68640cec00 msgr2=0x7f68641602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.730+0000 7f68690fd700 1 --2- 172.21.15.79:0/863396273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68640cec00 0x7f68641602e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6854009840 tx=0x7f6854009b50 comp rx=0 tx=0).stop 2023-06-23T18:37:10.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.730+0000 7f68690fd700 1 -- 172.21.15.79:0/863396273 shutdown_connections 2023-06-23T18:37:10.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.730+0000 7f68690fd700 1 --2- 172.21.15.79:0/863396273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68640cec00 0x7f68641602e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.730+0000 7f68690fd700 1 -- 172.21.15.79:0/863396273 >> 172.21.15.79:0/863396273 conn(0x7f6864089fc0 msgr2=0x7f6864099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:10.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.730+0000 7f68690fd700 1 -- 172.21.15.79:0/863396273 shutdown_connections 2023-06-23T18:37:10.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.730+0000 7f68690fd700 1 -- 172.21.15.79:0/863396273 wait complete. 2023-06-23T18:37:10.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.731+0000 7f68690fd700 1 Processor -- start 2023-06-23T18:37:10.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.731+0000 7f68690fd700 1 -- start start 2023-06-23T18:37:10.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.732+0000 7f68690fd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68640b17d0 0x7f68641532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:10.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.732+0000 7f68690fd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68640cec00 0x7f68640b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:10.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.732+0000 7f68690fd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f686415fa60 0x7f68640cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:10.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.732+0000 7f68690fd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6864153800 con 0x7f68640cec00 2023-06-23T18:37:10.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.732+0000 7f68690fd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68640ad650 con 0x7f686415fa60 2023-06-23T18:37:10.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.732+0000 7f68690fd700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68640ad8f0 con 0x7f68640b17d0 2023-06-23T18:37:10.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.732+0000 7f686259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68640cec00 0x7f68640b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:10.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.732+0000 7f686259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68640cec00 0x7f68640b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41854/0 (socket says 172.21.15.79:41854) 2023-06-23T18:37:10.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.732+0000 7f686259c700 1 -- 172.21.15.79:0/567213266 learned_addr learned my addr 172.21.15.79:0/567213266 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:10.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.732+0000 7f686359e700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f686415fa60 0x7f68640cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:10.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.732+0000 7f6862d9d700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68640b17d0 0x7f68641532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:10.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.733+0000 7f686259c700 1 -- 172.21.15.79:0/567213266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68640b17d0 msgr2=0x7f68641532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.733+0000 7f686259c700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68640b17d0 0x7f68641532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.733+0000 7f686259c700 1 -- 172.21.15.79:0/567213266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f686415fa60 msgr2=0x7f68640cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.733+0000 7f686259c700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f686415fa60 0x7f68640cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.733+0000 7f686259c700 1 -- 172.21.15.79:0/567213266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68540094a0 con 0x7f68640cec00 2023-06-23T18:37:10.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.733+0000 7f6862d9d700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68640b17d0 0x7f68641532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:10.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.733+0000 7f686259c700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68640cec00 0x7f68640b6090 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6854004000 tx=0x7f6854004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:10.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.733+0000 7f684b7fe700 1 -- 172.21.15.79:0/567213266 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854003e90 con 0x7f68640cec00 2023-06-23T18:37:10.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.733+0000 7f684b7fe700 1 -- 172.21.15.79:0/567213266 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685401e070 con 0x7f68640cec00 2023-06-23T18:37:10.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.733+0000 7f68690fd700 1 -- 172.21.15.79:0/567213266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68640adbd0 con 0x7f68640cec00 2023-06-23T18:37:10.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.733+0000 7f684b7fe700 1 -- 172.21.15.79:0/567213266 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854011850 con 0x7f68640cec00 2023-06-23T18:37:10.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.733+0000 7f68690fd700 1 -- 172.21.15.79:0/567213266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68640ae190 con 0x7f68640cec00 2023-06-23T18:37:10.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.734+0000 7f68690fd700 1 -- 172.21.15.79:0/567213266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686406a7b0 con 0x7f68640cec00 2023-06-23T18:37:10.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.734+0000 7f684b7fe700 1 -- 172.21.15.79:0/567213266 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f68540119b0 con 0x7f68640cec00 2023-06-23T18:37:10.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.735+0000 7f684b7fe700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f685006ace0 0x7f685006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:10.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.735+0000 7f684b7fe700 1 -- 172.21.15.79:0/567213266 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f685408a760 con 0x7f68640cec00 2023-06-23T18:37:10.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.735+0000 7f6862d9d700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f685006ace0 0x7f685006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:10.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.735+0000 7f6862d9d700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f685006ace0 0x7f685006d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6858006fd0 tx=0x7f6858008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:10.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.739+0000 7f684b7fe700 1 -- 172.21.15.79:0/567213266 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f68540558a0 con 0x7f68640cec00 2023-06-23T18:37:10.932 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.927+0000 7f68690fd700 1 -- 172.21.15.79:0/567213266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68640ae380 con 0x7f68640cec00 2023-06-23T18:37:10.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.928+0000 7f684b7fe700 1 -- 172.21.15.79:0/567213266 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+18305 (secure 0 0 0) 0x7f6854058ec0 con 0x7f68640cec00 2023-06-23T18:37:10.933 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:10.934 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":34,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:36:59.547297+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:36:56.964534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-23T18:36:57.507393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:37:10.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.930+0000 7f68690fd700 1 -- 172.21.15.79:0/567213266 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f685006ace0 msgr2=0x7f685006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.930+0000 7f68690fd700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f685006ace0 0x7f685006d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6858006fd0 tx=0x7f6858008040 comp rx=0 tx=0).stop 2023-06-23T18:37:10.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.930+0000 7f68690fd700 1 -- 172.21.15.79:0/567213266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68640cec00 msgr2=0x7f68640b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:10.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.930+0000 7f68690fd700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68640cec00 0x7f68640b6090 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6854004000 tx=0x7f6854004750 comp rx=0 tx=0).stop 2023-06-23T18:37:10.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.930+0000 7f68690fd700 1 -- 172.21.15.79:0/567213266 shutdown_connections 2023-06-23T18:37:10.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.930+0000 7f68690fd700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f685006ace0 0x7f685006d190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.930+0000 7f68690fd700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f686415fa60 0x7f68640cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.930+0000 7f68690fd700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68640b17d0 0x7f68641532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.930+0000 7f68690fd700 1 --2- 172.21.15.79:0/567213266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68640cec00 0x7f68640b6090 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:10.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.930+0000 7f68690fd700 1 -- 172.21.15.79:0/567213266 >> 172.21.15.79:0/567213266 conn(0x7f6864089fc0 msgr2=0x7f686415b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:10.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.930+0000 7f68690fd700 1 -- 172.21.15.79:0/567213266 shutdown_connections 2023-06-23T18:37:10.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:10.930+0000 7f68690fd700 1 -- 172.21.15.79:0/567213266 wait complete. 2023-06-23T18:37:10.947 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:36:56.964534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:36:57.507393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:37:10.948 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:37:11.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.066+0000 7f37988ff700 1 Processor -- start 2023-06-23T18:37:11.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.066+0000 7f37988ff700 1 -- start start 2023-06-23T18:37:11.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.067+0000 7f37988ff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37940cb870 0x7f3794160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.067+0000 7f37988ff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f37940cf1f0 0x7f379415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.067+0000 7f37988ff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3794069e90 0x7f379406a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.067+0000 7f37988ff700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f379415cac0 con 0x7f37940cf1f0 2023-06-23T18:37:11.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.067+0000 7f37988ff700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f379406a850 con 0x7f3794069e90 2023-06-23T18:37:11.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.067+0000 7f37988ff700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3794160f50 con 0x7f37940cb870 2023-06-23T18:37:11.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.068+0000 7f3791d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f37940cf1f0 0x7f379415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.068+0000 7f3791d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f37940cf1f0 0x7f379415d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41862/0 (socket says 172.21.15.79:41862) 2023-06-23T18:37:11.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.068+0000 7f3791d9b700 1 -- 172.21.15.79:0/1468210074 learned_addr learned my addr 172.21.15.79:0/1468210074 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:11.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.068+0000 7f379259c700 1 --2- 172.21.15.79:0/1468210074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37940cb870 0x7f3794160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.068+0000 7f3791d9b700 1 -- 172.21.15.79:0/1468210074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37940cb870 msgr2=0x7f3794160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:11.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.068+0000 7f3792d9d700 1 --2- 172.21.15.79:0/1468210074 >> v2:172.21.15.155:3300/0 conn(0x7f3794069e90 0x7f379406a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.068+0000 7f3791d9b700 1 --2- 172.21.15.79:0/1468210074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37940cb870 0x7f3794160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.068+0000 7f3791d9b700 1 -- 172.21.15.79:0/1468210074 >> v2:172.21.15.155:3300/0 conn(0x7f3794069e90 msgr2=0x7f379406a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:11.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.068+0000 7f3791d9b700 1 --2- 172.21.15.79:0/1468210074 >> v2:172.21.15.155:3300/0 conn(0x7f3794069e90 0x7f379406a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.068+0000 7f3791d9b700 1 -- 172.21.15.79:0/1468210074 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37940cd410 con 0x7f37940cf1f0 2023-06-23T18:37:11.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.068+0000 7f3791d9b700 1 --2- 172.21.15.79:0/1468210074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37940cf1f0 0x7f379415d4d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f37940743b0 tx=0x7f3784009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58d2482b47e5e49 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:11.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.068+0000 7f379159a700 1 -- 172.21.15.79:0/1468210074 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400b870 con 0x7f37940cf1f0 2023-06-23T18:37:11.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.068+0000 7f379159a700 1 -- 172.21.15.79:0/1468210074 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378400b9d0 con 0x7f37940cf1f0 2023-06-23T18:37:11.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.068+0000 7f379159a700 1 -- 172.21.15.79:0/1468210074 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400bd60 con 0x7f37940cf1f0 2023-06-23T18:37:11.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.069+0000 7f37988ff700 1 -- 172.21.15.79:0/1468210074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37940cf1f0 msgr2=0x7f379415d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.069+0000 7f37988ff700 1 --2- 172.21.15.79:0/1468210074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37940cf1f0 0x7f379415d4d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f37940743b0 tx=0x7f3784009bd0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.069+0000 7f37988ff700 1 -- 172.21.15.79:0/1468210074 shutdown_connections 2023-06-23T18:37:11.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.069+0000 7f37988ff700 1 --2- 172.21.15.79:0/1468210074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37940cb870 0x7f3794160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.069+0000 7f37988ff700 1 --2- 172.21.15.79:0/1468210074 >> v2:172.21.15.155:3300/0 conn(0x7f3794069e90 0x7f379406a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.069+0000 7f37988ff700 1 --2- 172.21.15.79:0/1468210074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37940cf1f0 0x7f379415d4d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.069+0000 7f37988ff700 1 -- 172.21.15.79:0/1468210074 >> 172.21.15.79:0/1468210074 conn(0x7f379406e200 msgr2=0x7f37940cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:11.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.069+0000 7f37988ff700 1 -- 172.21.15.79:0/1468210074 shutdown_connections 2023-06-23T18:37:11.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.069+0000 7f37988ff700 1 -- 172.21.15.79:0/1468210074 wait complete. 2023-06-23T18:37:11.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.070+0000 7f37988ff700 1 Processor -- start 2023-06-23T18:37:11.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.070+0000 7f37988ff700 1 -- start start 2023-06-23T18:37:11.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f37988ff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3794069e90 0x7f3794151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f37988ff700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37940cb870 0x7f3794160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f37988ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37940cf1f0 0x7f37941605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f37988ff700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37941f3870 con 0x7f3794069e90 2023-06-23T18:37:11.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f37988ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379415df10 con 0x7f37940cf1f0 2023-06-23T18:37:11.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f37988ff700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3794152340 con 0x7f37940cb870 2023-06-23T18:37:11.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f379259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3794069e90 0x7f3794151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f379259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3794069e90 0x7f3794151e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41866/0 (socket says 172.21.15.79:41866) 2023-06-23T18:37:11.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f379259c700 1 -- 172.21.15.79:0/2607723075 learned_addr learned my addr 172.21.15.79:0/2607723075 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:11.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f3791d9b700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37940cb870 0x7f3794160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f379259c700 1 -- 172.21.15.79:0/2607723075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37940cb870 msgr2=0x7f3794160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f3792d9d700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37940cf1f0 0x7f37941605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f379259c700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37940cb870 0x7f3794160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f379259c700 1 -- 172.21.15.79:0/2607723075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37940cf1f0 msgr2=0x7f37941605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f379259c700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37940cf1f0 0x7f37941605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f379259c700 1 -- 172.21.15.79:0/2607723075 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3784009550 con 0x7f3794069e90 2023-06-23T18:37:11.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f3792d9d700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37940cf1f0 0x7f37941605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:11.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.071+0000 7f379259c700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3794069e90 0x7f3794151e00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f378800c7f0 tx=0x7f378800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:11.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.072+0000 7f3782ffd700 1 -- 172.21.15.79:0/2607723075 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788011800 con 0x7f3794069e90 2023-06-23T18:37:11.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.072+0000 7f3782ffd700 1 -- 172.21.15.79:0/2607723075 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3788011960 con 0x7f3794069e90 2023-06-23T18:37:11.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.072+0000 7f37988ff700 1 -- 172.21.15.79:0/2607723075 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379415dbc0 con 0x7f3794069e90 2023-06-23T18:37:11.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.072+0000 7f3782ffd700 1 -- 172.21.15.79:0/2607723075 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788011cf0 con 0x7f3794069e90 2023-06-23T18:37:11.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.072+0000 7f37988ff700 1 -- 172.21.15.79:0/2607723075 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37940c43e0 con 0x7f3794069e90 2023-06-23T18:37:11.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.073+0000 7f37988ff700 1 -- 172.21.15.79:0/2607723075 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379415d9f0 con 0x7f3794069e90 2023-06-23T18:37:11.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.073+0000 7f3782ffd700 1 -- 172.21.15.79:0/2607723075 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3788018360 con 0x7f3794069e90 2023-06-23T18:37:11.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.073+0000 7f3782ffd700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f377406ace0 0x7f377406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.074+0000 7f3782ffd700 1 -- 172.21.15.79:0/2607723075 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3788089420 con 0x7f3794069e90 2023-06-23T18:37:11.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.074+0000 7f3791d9b700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f377406ace0 0x7f377406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.074+0000 7f3791d9b700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f377406ace0 0x7f377406d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f37940b46b0 tx=0x7f3784008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:11.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.078+0000 7f3782ffd700 1 -- 172.21.15.79:0/2607723075 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f379415d9f0 con 0x7f3794069e90 2023-06-23T18:37:11.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.264+0000 7f37988ff700 1 -- 172.21.15.79:0/2607723075 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f379415d9f0 con 0x7f3794069e90 2023-06-23T18:37:11.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.265+0000 7f3782ffd700 1 -- 172.21.15.79:0/2607723075 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7f379415d9f0 con 0x7f3794069e90 2023-06-23T18:37:11.270 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:37:11.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.267+0000 7f37988ff700 1 -- 172.21.15.79:0/2607723075 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f377406ace0 msgr2=0x7f377406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.267+0000 7f37988ff700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f377406ace0 0x7f377406d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f37940b46b0 tx=0x7f3784008030 comp rx=0 tx=0).stop 2023-06-23T18:37:11.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.267+0000 7f37988ff700 1 -- 172.21.15.79:0/2607723075 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3794069e90 msgr2=0x7f3794151e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.267+0000 7f37988ff700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3794069e90 0x7f3794151e00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f378800c7f0 tx=0x7f378800cbb0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.267+0000 7f37988ff700 1 -- 172.21.15.79:0/2607723075 shutdown_connections 2023-06-23T18:37:11.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.267+0000 7f37988ff700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f377406ace0 0x7f377406d190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.267+0000 7f37988ff700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37940cf1f0 0x7f37941605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.267+0000 7f37988ff700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3794069e90 0x7f3794151e00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.267+0000 7f37988ff700 1 --2- 172.21.15.79:0/2607723075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37940cb870 0x7f3794160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.267+0000 7f37988ff700 1 -- 172.21.15.79:0/2607723075 >> 172.21.15.79:0/2607723075 conn(0x7f379406e200 msgr2=0x7f37940b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:11.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.267+0000 7f37988ff700 1 -- 172.21.15.79:0/2607723075 shutdown_connections 2023-06-23T18:37:11.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.267+0000 7f37988ff700 1 -- 172.21.15.79:0/2607723075 wait complete. 2023-06-23T18:37:11.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:37:11.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.403+0000 7fbfc992e700 1 Processor -- start 2023-06-23T18:37:11.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.403+0000 7fbfc992e700 1 -- start start 2023-06-23T18:37:11.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc992e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc415c420 0x7fbfc4070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc992e700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbfc40caf40 0x7fbfc40ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc992e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbfc406c8b0 0x7fbfc415fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc992e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc406b9b0 con 0x7fbfc406c8b0 2023-06-23T18:37:11.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc992e700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc406c710 con 0x7fbfc40caf40 2023-06-23T18:37:11.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc992e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc406bf00 con 0x7fbfc415c420 2023-06-23T18:37:11.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc37fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbfc406c8b0 0x7fbfc415fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc37fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbfc406c8b0 0x7fbfc415fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41878/0 (socket says 172.21.15.79:41878) 2023-06-23T18:37:11.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc37fe700 1 -- 172.21.15.79:0/3622038052 learned_addr learned my addr 172.21.15.79:0/3622038052 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:11.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc27fc700 1 --2- 172.21.15.79:0/3622038052 >> v2:172.21.15.155:3300/0 conn(0x7fbfc40caf40 0x7fbfc40ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc2ffd700 1 --2- 172.21.15.79:0/3622038052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc415c420 0x7fbfc4070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc37fe700 1 -- 172.21.15.79:0/3622038052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc415c420 msgr2=0x7fbfc4070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:11.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc37fe700 1 --2- 172.21.15.79:0/3622038052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc415c420 0x7fbfc4070af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc37fe700 1 -- 172.21.15.79:0/3622038052 >> v2:172.21.15.155:3300/0 conn(0x7fbfc40caf40 msgr2=0x7fbfc40ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:11.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc37fe700 1 --2- 172.21.15.79:0/3622038052 >> v2:172.21.15.155:3300/0 conn(0x7fbfc40caf40 0x7fbfc40ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc37fe700 1 -- 172.21.15.79:0/3622038052 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc40cb3b0 con 0x7fbfc406c8b0 2023-06-23T18:37:11.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc2ffd700 1 --2- 172.21.15.79:0/3622038052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc415c420 0x7fbfc4070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:11.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc37fe700 1 --2- 172.21.15.79:0/3622038052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc406c8b0 0x7fbfc415fd90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40b34b0 tx=0x7fbfbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57ba8503ce2a01e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:11.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc1ffb700 1 -- 172.21.15.79:0/3622038052 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00e070 con 0x7fbfc406c8b0 2023-06-23T18:37:11.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc1ffb700 1 -- 172.21.15.79:0/3622038052 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc00b9c0 con 0x7fbfc406c8b0 2023-06-23T18:37:11.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.404+0000 7fbfc1ffb700 1 -- 172.21.15.79:0/3622038052 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00bd70 con 0x7fbfc406c8b0 2023-06-23T18:37:11.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.405+0000 7fbfc992e700 1 -- 172.21.15.79:0/3622038052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc406c8b0 msgr2=0x7fbfc415fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.405+0000 7fbfc992e700 1 --2- 172.21.15.79:0/3622038052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc406c8b0 0x7fbfc415fd90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40b34b0 tx=0x7fbfbc00b330 comp rx=0 tx=0).stop 2023-06-23T18:37:11.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.405+0000 7fbfc992e700 1 -- 172.21.15.79:0/3622038052 shutdown_connections 2023-06-23T18:37:11.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.405+0000 7fbfc992e700 1 --2- 172.21.15.79:0/3622038052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc415c420 0x7fbfc4070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.405+0000 7fbfc992e700 1 --2- 172.21.15.79:0/3622038052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc406c8b0 0x7fbfc415fd90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.405+0000 7fbfc992e700 1 --2- 172.21.15.79:0/3622038052 >> v2:172.21.15.155:3300/0 conn(0x7fbfc40caf40 0x7fbfc40ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.405+0000 7fbfc992e700 1 -- 172.21.15.79:0/3622038052 >> 172.21.15.79:0/3622038052 conn(0x7fbfc406e540 msgr2=0x7fbfc406e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:11.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.405+0000 7fbfc992e700 1 -- 172.21.15.79:0/3622038052 shutdown_connections 2023-06-23T18:37:11.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.405+0000 7fbfc992e700 1 -- 172.21.15.79:0/3622038052 wait complete. 2023-06-23T18:37:11.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.407+0000 7fbfc992e700 1 Processor -- start 2023-06-23T18:37:11.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.407+0000 7fbfc992e700 1 -- start start 2023-06-23T18:37:11.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.407+0000 7fbfc992e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc406c8b0 0x7fbfc4153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.407+0000 7fbfc992e700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc40caf40 0x7fbfc415c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.407+0000 7fbfc992e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc415c420 0x7fbfc40b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.407+0000 7fbfc992e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc415f4e0 con 0x7fbfc415c420 2023-06-23T18:37:11.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.407+0000 7fbfc992e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc40b45a0 con 0x7fbfc406c8b0 2023-06-23T18:37:11.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.407+0000 7fbfc992e700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc415d020 con 0x7fbfc40caf40 2023-06-23T18:37:11.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfc37fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc415c420 0x7fbfc40b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfc37fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc415c420 0x7fbfc40b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41884/0 (socket says 172.21.15.79:41884) 2023-06-23T18:37:11.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfc37fe700 1 -- 172.21.15.79:0/2088367656 learned_addr learned my addr 172.21.15.79:0/2088367656 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:11.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfc2ffd700 1 --2- 172.21.15.79:0/2088367656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc406c8b0 0x7fbfc4153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfc27fc700 1 --2- 172.21.15.79:0/2088367656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc40caf40 0x7fbfc415c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfc37fe700 1 -- 172.21.15.79:0/2088367656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc40caf40 msgr2=0x7fbfc415c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfc37fe700 1 --2- 172.21.15.79:0/2088367656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc40caf40 0x7fbfc415c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfc37fe700 1 -- 172.21.15.79:0/2088367656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc406c8b0 msgr2=0x7fbfc4153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfc37fe700 1 --2- 172.21.15.79:0/2088367656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc406c8b0 0x7fbfc4153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfc37fe700 1 -- 172.21.15.79:0/2088367656 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc009b30 con 0x7fbfc415c420 2023-06-23T18:37:11.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfc37fe700 1 --2- 172.21.15.79:0/2088367656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc415c420 0x7fbfc40b5e90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00b370 tx=0x7fbfbc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:11.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfabfff700 1 -- 172.21.15.79:0/2088367656 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc004250 con 0x7fbfc415c420 2023-06-23T18:37:11.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfabfff700 1 -- 172.21.15.79:0/2088367656 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc0043b0 con 0x7fbfc415c420 2023-06-23T18:37:11.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfc992e700 1 -- 172.21.15.79:0/2088367656 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc415d300 con 0x7fbfc415c420 2023-06-23T18:37:11.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfabfff700 1 -- 172.21.15.79:0/2088367656 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc004760 con 0x7fbfc415c420 2023-06-23T18:37:11.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.408+0000 7fbfc992e700 1 -- 172.21.15.79:0/2088367656 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc415d8c0 con 0x7fbfc415c420 2023-06-23T18:37:11.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.409+0000 7fbfc992e700 1 -- 172.21.15.79:0/2088367656 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc40b4710 con 0x7fbfc415c420 2023-06-23T18:37:11.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.410+0000 7fbfabfff700 1 -- 172.21.15.79:0/2088367656 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbfbc01b430 con 0x7fbfc415c420 2023-06-23T18:37:11.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.410+0000 7fbfabfff700 1 --2- 172.21.15.79:0/2088367656 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbfb006ace0 0x7fbfb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.410+0000 7fbfabfff700 1 -- 172.21.15.79:0/2088367656 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbfbc08a5f0 con 0x7fbfc415c420 2023-06-23T18:37:11.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.410+0000 7fbfc2ffd700 1 --2- 172.21.15.79:0/2088367656 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbfb006ace0 0x7fbfb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.410+0000 7fbfc2ffd700 1 --2- 172.21.15.79:0/2088367656 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbfb006ace0 0x7fbfb006d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8006fd0 tx=0x7fbfb8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:11.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.414+0000 7fbfabfff700 1 -- 172.21.15.79:0/2088367656 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbfbc055730 con 0x7fbfc415c420 2023-06-23T18:37:11.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.599+0000 7fbfc992e700 1 -- 172.21.15.79:0/2088367656 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbfc404e1d0 con 0x7fbfc415c420 2023-06-23T18:37:11.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.600+0000 7fbfabfff700 1 -- 172.21.15.79:0/2088367656 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbfbc058d50 con 0x7fbfc415c420 2023-06-23T18:37:11.605 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:37:11.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.602+0000 7fbfc992e700 1 -- 172.21.15.79:0/2088367656 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbfb006ace0 msgr2=0x7fbfb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.602+0000 7fbfc992e700 1 --2- 172.21.15.79:0/2088367656 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbfb006ace0 0x7fbfb006d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8006fd0 tx=0x7fbfb8008040 comp rx=0 tx=0).stop 2023-06-23T18:37:11.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.602+0000 7fbfc992e700 1 -- 172.21.15.79:0/2088367656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc415c420 msgr2=0x7fbfc40b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.602+0000 7fbfc992e700 1 --2- 172.21.15.79:0/2088367656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc415c420 0x7fbfc40b5e90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00b370 tx=0x7fbfbc004ad0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.602+0000 7fbfc992e700 1 -- 172.21.15.79:0/2088367656 shutdown_connections 2023-06-23T18:37:11.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.602+0000 7fbfc992e700 1 --2- 172.21.15.79:0/2088367656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc415c420 0x7fbfc40b5e90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.602+0000 7fbfc992e700 1 --2- 172.21.15.79:0/2088367656 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbfb006ace0 0x7fbfb006d190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.602+0000 7fbfc992e700 1 --2- 172.21.15.79:0/2088367656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc406c8b0 0x7fbfc4153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.602+0000 7fbfc992e700 1 --2- 172.21.15.79:0/2088367656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc40caf40 0x7fbfc415c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.602+0000 7fbfc992e700 1 -- 172.21.15.79:0/2088367656 >> 172.21.15.79:0/2088367656 conn(0x7fbfc406e540 msgr2=0x7fbfc40cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:11.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.602+0000 7fbfc992e700 1 -- 172.21.15.79:0/2088367656 shutdown_connections 2023-06-23T18:37:11.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.602+0000 7fbfc992e700 1 -- 172.21.15.79:0/2088367656 wait complete. 2023-06-23T18:37:11.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:37:11.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.737+0000 7f348442d700 1 Processor -- start 2023-06-23T18:37:11.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.737+0000 7f348442d700 1 -- start start 2023-06-23T18:37:11.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.738+0000 7f348442d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f347c0cedf0 0x7f347c0b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.738+0000 7f348442d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f347c0b22c0 con 0x7f347c0d08c0 2023-06-23T18:37:11.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.738+0000 7f348442d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f347c0b0b10 con 0x7f347c1438f0 2023-06-23T18:37:11.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.738+0000 7f348442d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347c0b0df0 con 0x7f347c0cedf0 2023-06-23T18:37:11.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.738+0000 7f34829ca700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f347c0d08c0 0x7f347c0ad530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44934/0 (socket says 172.21.15.79:44934) 2023-06-23T18:37:11.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.738+0000 7f34829ca700 1 -- 172.21.15.79:0/1717943009 learned_addr learned my addr 172.21.15.79:0/1717943009 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:11.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.738+0000 7f34821c9700 1 --2- 172.21.15.79:0/1717943009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f347c0cedf0 0x7f347c0b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.738+0000 7f34811c7700 1 -- 172.21.15.79:0/1717943009 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670702938 0 0) 0x7f347c0b22c0 con 0x7f347c0d08c0 2023-06-23T18:37:11.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.738+0000 7f34811c7700 1 -- 172.21.15.79:0/1717943009 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34680035b0 con 0x7f347c0d08c0 2023-06-23T18:37:11.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.739+0000 7f34811c7700 1 -- 172.21.15.79:0/1717943009 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922324313 0 0) 0x7f347c0b0b10 con 0x7f347c1438f0 2023-06-23T18:37:11.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.739+0000 7f34811c7700 1 -- 172.21.15.79:0/1717943009 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f347c0b22c0 con 0x7f347c1438f0 2023-06-23T18:37:11.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.739+0000 7f34821c9700 1 -- 172.21.15.79:0/1717943009 >> v1:172.21.15.155:6789/0 conn(0x7f347c1438f0 legacy=0x7f347c0b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.739+0000 7f34821c9700 1 -- 172.21.15.79:0/1717943009 >> v1:172.21.15.79:6789/0 conn(0x7f347c0d08c0 legacy=0x7f347c0ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.739+0000 7f34811c7700 1 -- 172.21.15.79:0/1717943009 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1526874897 0 0) 0x7f34680035b0 con 0x7f347c0d08c0 2023-06-23T18:37:11.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.739+0000 7f34821c9700 1 -- 172.21.15.79:0/1717943009 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c0b10d0 con 0x7f347c0cedf0 2023-06-23T18:37:11.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.739+0000 7f34821c9700 1 --2- 172.21.15.79:0/1717943009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f347c0cedf0 0x7f347c0b02c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f346c009850 tx=0x7f346c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d936401d0c80cead server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:11.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.739+0000 7f34811c7700 1 -- 172.21.15.79:0/1717943009 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34740019f0 con 0x7f347c0cedf0 2023-06-23T18:37:11.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.739+0000 7f34811c7700 1 -- 172.21.15.79:0/1717943009 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346c005c30 con 0x7f347c0cedf0 2023-06-23T18:37:11.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.739+0000 7f34811c7700 1 -- 172.21.15.79:0/1717943009 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c004030 con 0x7f347c0cedf0 2023-06-23T18:37:11.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.740+0000 7f348442d700 1 -- 172.21.15.79:0/1717943009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f347c0cedf0 msgr2=0x7f347c0b02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.740+0000 7f348442d700 1 --2- 172.21.15.79:0/1717943009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f347c0cedf0 0x7f347c0b02c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f346c009850 tx=0x7f346c009b60 comp rx=0 tx=0).stop 2023-06-23T18:37:11.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.740+0000 7f348442d700 1 -- 172.21.15.79:0/1717943009 shutdown_connections 2023-06-23T18:37:11.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.740+0000 7f348442d700 1 --2- 172.21.15.79:0/1717943009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f347c0cedf0 0x7f347c0b02c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.740+0000 7f348442d700 1 -- 172.21.15.79:0/1717943009 >> 172.21.15.79:0/1717943009 conn(0x7f347c06a7b0 msgr2=0x7f347c099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:11.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.740+0000 7f348442d700 1 -- 172.21.15.79:0/1717943009 shutdown_connections 2023-06-23T18:37:11.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.740+0000 7f348442d700 1 -- 172.21.15.79:0/1717943009 wait complete. 2023-06-23T18:37:11.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.741+0000 7f348442d700 1 Processor -- start 2023-06-23T18:37:11.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.741+0000 7f348442d700 1 -- start start 2023-06-23T18:37:11.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f348442d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f347c0cedf0 0x7f347c1575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f348442d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f347c0d08c0 0x7f347c156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f348442d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f347c1438f0 0x7f347c0c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f348442d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c157c00 con 0x7f347c0d08c0 2023-06-23T18:37:11.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f348442d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c157ee0 con 0x7f347c1438f0 2023-06-23T18:37:11.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f348442d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347c1581c0 con 0x7f347c0cedf0 2023-06-23T18:37:11.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f34819c8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f347c0d08c0 0x7f347c156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f34819c8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f347c0d08c0 0x7f347c156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41896/0 (socket says 172.21.15.79:41896) 2023-06-23T18:37:11.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f34819c8700 1 -- 172.21.15.79:0/2451511991 learned_addr learned my addr 172.21.15.79:0/2451511991 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:11.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f34829ca700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f347c1438f0 0x7f347c0c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f34821c9700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f347c0cedf0 0x7f347c1575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f34819c8700 1 -- 172.21.15.79:0/2451511991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f347c0cedf0 msgr2=0x7f347c1575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f34819c8700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f347c0cedf0 0x7f347c1575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f34819c8700 1 -- 172.21.15.79:0/2451511991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f347c1438f0 msgr2=0x7f347c0c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f34819c8700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f347c1438f0 0x7f347c0c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f34819c8700 1 -- 172.21.15.79:0/2451511991 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c009430 con 0x7f347c0d08c0 2023-06-23T18:37:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f34829ca700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f347c1438f0 0x7f347c0c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f34821c9700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f347c0cedf0 0x7f347c1575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f34819c8700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f347c0d08c0 0x7f347c156600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f347800c500 tx=0x7f347800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f3472ffd700 1 -- 172.21.15.79:0/2451511991 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34780052d0 con 0x7f347c0d08c0 2023-06-23T18:37:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f3472ffd700 1 -- 172.21.15.79:0/2451511991 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3478005450 con 0x7f347c0d08c0 2023-06-23T18:37:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f348442d700 1 -- 172.21.15.79:0/2451511991 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347c15a800 con 0x7f347c0d08c0 2023-06-23T18:37:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f3472ffd700 1 -- 172.21.15.79:0/2451511991 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34780038c0 con 0x7f347c0d08c0 2023-06-23T18:37:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.742+0000 7f348442d700 1 -- 172.21.15.79:0/2451511991 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347c0c61a0 con 0x7f347c0d08c0 2023-06-23T18:37:11.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.743+0000 7f348442d700 1 -- 172.21.15.79:0/2451511991 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347c15aa40 con 0x7f347c0d08c0 2023-06-23T18:37:11.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.744+0000 7f3472ffd700 1 -- 172.21.15.79:0/2451511991 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3478003a20 con 0x7f347c0d08c0 2023-06-23T18:37:11.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.744+0000 7f3472ffd700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f346006ace0 0x7f346006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:11.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.744+0000 7f3472ffd700 1 -- 172.21.15.79:0/2451511991 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3478088970 con 0x7f347c0d08c0 2023-06-23T18:37:11.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.744+0000 7f34821c9700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f346006ace0 0x7f346006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:11.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.744+0000 7f34821c9700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f346006ace0 0x7f346006d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f346c014040 tx=0x7f346c008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:11.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.748+0000 7f3472ffd700 1 -- 172.21.15.79:0/2451511991 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f347c15aa40 con 0x7f347c0d08c0 2023-06-23T18:37:11.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.935+0000 7f348442d700 1 -- 172.21.15.79:0/2451511991 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f347c15aa40 con 0x7f347c0d08c0 2023-06-23T18:37:11.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.935+0000 7f3472ffd700 1 -- 172.21.15.79:0/2451511991 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f347c15aa40 con 0x7f347c0d08c0 2023-06-23T18:37:11.941 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:37:11.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.937+0000 7f348442d700 1 -- 172.21.15.79:0/2451511991 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f346006ace0 msgr2=0x7f346006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.937+0000 7f348442d700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f346006ace0 0x7f346006d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f346c014040 tx=0x7f346c008070 comp rx=0 tx=0).stop 2023-06-23T18:37:11.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.937+0000 7f348442d700 1 -- 172.21.15.79:0/2451511991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f347c0d08c0 msgr2=0x7f347c156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:11.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.937+0000 7f348442d700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f347c0d08c0 0x7f347c156600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f347800c500 tx=0x7f347800c8c0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.937+0000 7f348442d700 1 -- 172.21.15.79:0/2451511991 shutdown_connections 2023-06-23T18:37:11.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.937+0000 7f348442d700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f346006ace0 0x7f346006d190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.937+0000 7f348442d700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f347c1438f0 0x7f347c0c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.937+0000 7f348442d700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f347c0cedf0 0x7f347c1575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.937+0000 7f348442d700 1 --2- 172.21.15.79:0/2451511991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f347c0d08c0 0x7f347c156600 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:11.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.937+0000 7f348442d700 1 -- 172.21.15.79:0/2451511991 >> 172.21.15.79:0/2451511991 conn(0x7f347c06a7b0 msgr2=0x7f347c0b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:11.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.937+0000 7f348442d700 1 -- 172.21.15.79:0/2451511991 shutdown_connections 2023-06-23T18:37:11.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:11.937+0000 7f348442d700 1 -- 172.21.15.79:0/2451511991 wait complete. 2023-06-23T18:37:11.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:37:12.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.072+0000 7fed38d84700 1 Processor -- start 2023-06-23T18:37:12.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.072+0000 7fed38d84700 1 -- start start 2023-06-23T18:37:12.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.073+0000 7fed38d84700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed3415cc50 0x7fed340cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:12.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.073+0000 7fed38d84700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fed3415d730 0x7fed340cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:12.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.073+0000 7fed38d84700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed340d2b10 con 0x7fed340d0760 2023-06-23T18:37:12.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.073+0000 7fed38d84700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed340d27e0 con 0x7fed3415d730 2023-06-23T18:37:12.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.073+0000 7fed38d84700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed340cd310 con 0x7fed3415cc50 2023-06-23T18:37:12.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.073+0000 7fed32d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fed3415d730 0x7fed340cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:12.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.073+0000 7fed31d9b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fed340d0760 0x7fed3415d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44944/0 (socket says 172.21.15.79:44944) 2023-06-23T18:37:12.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.073+0000 7fed31d9b700 1 -- 172.21.15.79:0/4293542770 learned_addr learned my addr 172.21.15.79:0/4293542770 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:12.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.073+0000 7fed32d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fed3415d730 0x7fed340cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41902/0 (socket says 172.21.15.79:41902) 2023-06-23T18:37:12.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.074+0000 7fed32d9d700 1 -- 172.21.15.79:0/4293542770 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed3415cc50 msgr2=0x7fed340cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:12.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.074+0000 7fed3259c700 1 --2- 172.21.15.79:0/4293542770 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed3415cc50 0x7fed340cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:12.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.074+0000 7fed32d9d700 1 --2- 172.21.15.79:0/4293542770 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed3415cc50 0x7fed340cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.074+0000 7fed3159a700 1 -- 172.21.15.79:0/4293542770 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1601553495 0 0) 0x7fed340d2b10 con 0x7fed340d0760 2023-06-23T18:37:12.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.074+0000 7fed32d9d700 1 -- 172.21.15.79:0/4293542770 >> v1:172.21.15.79:6789/0 conn(0x7fed340d0760 legacy=0x7fed3415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:12.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.074+0000 7fed32d9d700 1 -- 172.21.15.79:0/4293542770 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed340cd5f0 con 0x7fed3415d730 2023-06-23T18:37:12.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.074+0000 7fed3259c700 1 --2- 172.21.15.79:0/4293542770 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed3415cc50 0x7fed340cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:12.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.074+0000 7fed32d9d700 1 --2- 172.21.15.79:0/4293542770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed3415d730 0x7fed340cf570 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009d90 tx=0x7fed2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a728d601375317f1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:12.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.074+0000 7fed3159a700 1 -- 172.21.15.79:0/4293542770 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c00e070 con 0x7fed3415d730 2023-06-23T18:37:12.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.074+0000 7fed3159a700 1 -- 172.21.15.79:0/4293542770 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2c00bbd0 con 0x7fed3415d730 2023-06-23T18:37:12.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.074+0000 7fed3159a700 1 -- 172.21.15.79:0/4293542770 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c012430 con 0x7fed3415d730 2023-06-23T18:37:12.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.074+0000 7fed38d84700 1 -- 172.21.15.79:0/4293542770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed3415d730 msgr2=0x7fed340cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:12.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.074+0000 7fed38d84700 1 --2- 172.21.15.79:0/4293542770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed3415d730 0x7fed340cf570 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009d90 tx=0x7fed2c00b330 comp rx=0 tx=0).stop 2023-06-23T18:37:12.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.075+0000 7fed38d84700 1 -- 172.21.15.79:0/4293542770 shutdown_connections 2023-06-23T18:37:12.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.075+0000 7fed38d84700 1 --2- 172.21.15.79:0/4293542770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed3415d730 0x7fed340cf570 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.075+0000 7fed38d84700 1 --2- 172.21.15.79:0/4293542770 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed3415cc50 0x7fed340cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.075+0000 7fed38d84700 1 -- 172.21.15.79:0/4293542770 >> 172.21.15.79:0/4293542770 conn(0x7fed340715c0 msgr2=0x7fed340719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:12.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.075+0000 7fed38d84700 1 -- 172.21.15.79:0/4293542770 shutdown_connections 2023-06-23T18:37:12.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.075+0000 7fed38d84700 1 -- 172.21.15.79:0/4293542770 wait complete. 2023-06-23T18:37:12.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.076+0000 7fed38d84700 1 Processor -- start 2023-06-23T18:37:12.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.076+0000 7fed38d84700 1 -- start start 2023-06-23T18:37:12.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed38d84700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed340d0760 0x7fed34151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:12.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed38d84700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed3415cc50 0x7fed340cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:12.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed38d84700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed3415d730 0x7fed3415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:12.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed38d84700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed341f1e30 con 0x7fed340d0760 2023-06-23T18:37:12.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed38d84700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed341f2ff0 con 0x7fed3415d730 2023-06-23T18:37:12.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed38d84700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed341f2020 con 0x7fed3415cc50 2023-06-23T18:37:12.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed3259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed340d0760 0x7fed34151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:12.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed3259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed340d0760 0x7fed34151980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41908/0 (socket says 172.21.15.79:41908) 2023-06-23T18:37:12.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed3259c700 1 -- 172.21.15.79:0/2183435914 learned_addr learned my addr 172.21.15.79:0/2183435914 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:12.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed31d9b700 1 --2- 172.21.15.79:0/2183435914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed3415cc50 0x7fed340cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:12.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed32d9d700 1 --2- 172.21.15.79:0/2183435914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed3415d730 0x7fed3415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:12.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed3259c700 1 -- 172.21.15.79:0/2183435914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed3415cc50 msgr2=0x7fed340cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:12.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed3259c700 1 --2- 172.21.15.79:0/2183435914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed3415cc50 0x7fed340cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed3259c700 1 -- 172.21.15.79:0/2183435914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed3415d730 msgr2=0x7fed3415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:12.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed3259c700 1 --2- 172.21.15.79:0/2183435914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed3415d730 0x7fed3415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed3259c700 1 -- 172.21.15.79:0/2183435914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c009a70 con 0x7fed340d0760 2023-06-23T18:37:12.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed3259c700 1 --2- 172.21.15.79:0/2183435914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed340d0760 0x7fed34151980 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fed2400c750 tx=0x7fed2400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:12.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed22ffd700 1 -- 172.21.15.79:0/2183435914 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24011800 con 0x7fed340d0760 2023-06-23T18:37:12.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed22ffd700 1 -- 172.21.15.79:0/2183435914 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed24011960 con 0x7fed340d0760 2023-06-23T18:37:12.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed38d84700 1 -- 172.21.15.79:0/2183435914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed341f2300 con 0x7fed340d0760 2023-06-23T18:37:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed22ffd700 1 -- 172.21.15.79:0/2183435914 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24011cd0 con 0x7fed340d0760 2023-06-23T18:37:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.077+0000 7fed38d84700 1 -- 172.21.15.79:0/2183435914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed341f28c0 con 0x7fed340d0760 2023-06-23T18:37:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.078+0000 7fed38d84700 1 -- 172.21.15.79:0/2183435914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed340672c0 con 0x7fed340d0760 2023-06-23T18:37:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.079+0000 7fed22ffd700 1 -- 172.21.15.79:0/2183435914 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed24021350 con 0x7fed340d0760 2023-06-23T18:37:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.079+0000 7fed22ffd700 1 --2- 172.21.15.79:0/2183435914 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fed1406ace0 0x7fed1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.079+0000 7fed22ffd700 1 -- 172.21.15.79:0/2183435914 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fed24089430 con 0x7fed340d0760 2023-06-23T18:37:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.079+0000 7fed31d9b700 1 --2- 172.21.15.79:0/2183435914 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fed1406ace0 0x7fed1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:12.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.080+0000 7fed31d9b700 1 --2- 172.21.15.79:0/2183435914 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fed1406ace0 0x7fed1406d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fed340b1cf0 tx=0x7fed28007c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:12.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.083+0000 7fed22ffd700 1 -- 172.21.15.79:0/2183435914 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fed24057f60 con 0x7fed340d0760 2023-06-23T18:37:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.270+0000 7fed38d84700 1 -- 172.21.15.79:0/2183435914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fed340b5040 con 0x7fed340d0760 2023-06-23T18:37:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.270+0000 7fed22ffd700 1 -- 172.21.15.79:0/2183435914 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7fed24057af0 con 0x7fed340d0760 2023-06-23T18:37:12.275 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:37:12.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.272+0000 7fed38d84700 1 -- 172.21.15.79:0/2183435914 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fed1406ace0 msgr2=0x7fed1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:12.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.272+0000 7fed38d84700 1 --2- 172.21.15.79:0/2183435914 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fed1406ace0 0x7fed1406d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fed340b1cf0 tx=0x7fed28007c90 comp rx=0 tx=0).stop 2023-06-23T18:37:12.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.272+0000 7fed38d84700 1 -- 172.21.15.79:0/2183435914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed340d0760 msgr2=0x7fed34151980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:12.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.272+0000 7fed38d84700 1 --2- 172.21.15.79:0/2183435914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed340d0760 0x7fed34151980 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fed2400c750 tx=0x7fed2400cb10 comp rx=0 tx=0).stop 2023-06-23T18:37:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.272+0000 7fed38d84700 1 -- 172.21.15.79:0/2183435914 shutdown_connections 2023-06-23T18:37:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.272+0000 7fed38d84700 1 --2- 172.21.15.79:0/2183435914 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fed1406ace0 0x7fed1406d190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.272+0000 7fed38d84700 1 --2- 172.21.15.79:0/2183435914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed3415d730 0x7fed3415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.272+0000 7fed38d84700 1 --2- 172.21.15.79:0/2183435914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed340d0760 0x7fed34151980 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.272+0000 7fed38d84700 1 --2- 172.21.15.79:0/2183435914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed3415cc50 0x7fed340cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.272+0000 7fed38d84700 1 -- 172.21.15.79:0/2183435914 >> 172.21.15.79:0/2183435914 conn(0x7fed340715c0 msgr2=0x7fed340b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.272+0000 7fed38d84700 1 -- 172.21.15.79:0/2183435914 shutdown_connections 2023-06-23T18:37:12.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.272+0000 7fed38d84700 1 -- 172.21.15.79:0/2183435914 wait complete. 2023-06-23T18:37:12.289 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:37:12.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.404+0000 7f0d2bb40700 1 Processor -- start 2023-06-23T18:37:12.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.405+0000 7f0d2bb40700 1 -- start start 2023-06-23T18:37:12.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.405+0000 7f0d2bb40700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0d2415fa60 0x7f0d240ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:12.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.405+0000 7f0d2bb40700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d241609f0 0x7f0d240cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:12.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.405+0000 7f0d2bb40700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2415fe30 con 0x7f0d240b17d0 2023-06-23T18:37:12.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.405+0000 7f0d2bb40700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d24160e30 con 0x7f0d2415fa60 2023-06-23T18:37:12.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.405+0000 7f0d2bb40700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2406ce10 con 0x7f0d241609f0 2023-06-23T18:37:12.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.406+0000 7f0d298dc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0d2415fa60 0x7f0d240ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:12.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.406+0000 7f0d298dc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0d2415fa60 0x7f0d240ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41916/0 (socket says 172.21.15.79:41916) 2023-06-23T18:37:12.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.406+0000 7f0d298dc700 1 -- 172.21.15.79:0/211859531 learned_addr learned my addr 172.21.15.79:0/211859531 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:12.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.406+0000 7f0d2a0dd700 1 --2- 172.21.15.79:0/211859531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d241609f0 0x7f0d240cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:12.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.406+0000 7f0d298dc700 1 -- 172.21.15.79:0/211859531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d241609f0 msgr2=0x7f0d240cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:12.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.406+0000 7f0d298dc700 1 --2- 172.21.15.79:0/211859531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d241609f0 0x7f0d240cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.406+0000 7f0d298dc700 1 -- 172.21.15.79:0/211859531 >> v1:172.21.15.155:6789/0 conn(0x7f0d240b17d0 legacy=0x7f0d241602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:12.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.406+0000 7f0d298dc700 1 -- 172.21.15.79:0/211859531 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d240b0a10 con 0x7f0d2415fa60 2023-06-23T18:37:12.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.406+0000 7f0d298dc700 1 --2- 172.21.15.79:0/211859531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d2415fa60 0x7f0d240ce4f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0d14006fd0 tx=0x7f0d14009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f239f48aace75a22 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:12.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.406+0000 7f0d288da700 1 -- 172.21.15.79:0/211859531 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d14005e30 con 0x7f0d2415fa60 2023-06-23T18:37:12.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.406+0000 7f0d288da700 1 -- 172.21.15.79:0/211859531 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d14006030 con 0x7f0d2415fa60 2023-06-23T18:37:12.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.406+0000 7f0d288da700 1 -- 172.21.15.79:0/211859531 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d140063c0 con 0x7f0d2415fa60 2023-06-23T18:37:12.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.407+0000 7f0d2bb40700 1 -- 172.21.15.79:0/211859531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d2415fa60 msgr2=0x7f0d240ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:12.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.407+0000 7f0d2bb40700 1 --2- 172.21.15.79:0/211859531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d2415fa60 0x7f0d240ce4f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0d14006fd0 tx=0x7f0d14009bd0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.407+0000 7f0d2bb40700 1 -- 172.21.15.79:0/211859531 shutdown_connections 2023-06-23T18:37:12.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.407+0000 7f0d2bb40700 1 --2- 172.21.15.79:0/211859531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d2415fa60 0x7f0d240ce4f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.407+0000 7f0d2bb40700 1 --2- 172.21.15.79:0/211859531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d241609f0 0x7f0d240cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.407+0000 7f0d2bb40700 1 -- 172.21.15.79:0/211859531 >> 172.21.15.79:0/211859531 conn(0x7f0d24089fc0 msgr2=0x7f0d24099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:12.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.407+0000 7f0d2bb40700 1 -- 172.21.15.79:0/211859531 shutdown_connections 2023-06-23T18:37:12.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.407+0000 7f0d2bb40700 1 -- 172.21.15.79:0/211859531 wait complete. 2023-06-23T18:37:12.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.408+0000 7f0d2bb40700 1 Processor -- start 2023-06-23T18:37:12.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.408+0000 7f0d2bb40700 1 -- start start 2023-06-23T18:37:12.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d2bb40700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d240b17d0 0x7f0d240aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:12.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d2bb40700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d2415fa60 0x7f0d241f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:12.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d2bb40700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d241609f0 0x7f0d240c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:12.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d2bb40700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d240ae5f0 con 0x7f0d2415fa60 2023-06-23T18:37:12.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d2bb40700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d241f61b0 con 0x7f0d240b17d0 2023-06-23T18:37:12.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d2bb40700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d241f2d30 con 0x7f0d241609f0 2023-06-23T18:37:12.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d290db700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d2415fa60 0x7f0d241f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:12.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d290db700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d2415fa60 0x7f0d241f27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41924/0 (socket says 172.21.15.79:41924) 2023-06-23T18:37:12.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d290db700 1 -- 172.21.15.79:0/3586583285 learned_addr learned my addr 172.21.15.79:0/3586583285 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:12.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d298dc700 1 --2- 172.21.15.79:0/3586583285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d240b17d0 0x7f0d240aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:12.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d2a0dd700 1 --2- 172.21.15.79:0/3586583285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d241609f0 0x7f0d240c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:12.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d290db700 1 -- 172.21.15.79:0/3586583285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d241609f0 msgr2=0x7f0d240c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:12.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d290db700 1 --2- 172.21.15.79:0/3586583285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d241609f0 0x7f0d240c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d290db700 1 -- 172.21.15.79:0/3586583285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d240b17d0 msgr2=0x7f0d240aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:12.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d290db700 1 --2- 172.21.15.79:0/3586583285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d240b17d0 0x7f0d240aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d290db700 1 -- 172.21.15.79:0/3586583285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d14009500 con 0x7f0d2415fa60 2023-06-23T18:37:12.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d290db700 1 --2- 172.21.15.79:0/3586583285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d2415fa60 0x7f0d241f27f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000c4a0 tx=0x7f0d2000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:12.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.409+0000 7f0d1a7fc700 1 -- 172.21.15.79:0/3586583285 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20011790 con 0x7f0d2415fa60 2023-06-23T18:37:12.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.410+0000 7f0d1a7fc700 1 -- 172.21.15.79:0/3586583285 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d200118f0 con 0x7f0d2415fa60 2023-06-23T18:37:12.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.410+0000 7f0d2bb40700 1 -- 172.21.15.79:0/3586583285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d240ad970 con 0x7f0d2415fa60 2023-06-23T18:37:12.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.410+0000 7f0d1a7fc700 1 -- 172.21.15.79:0/3586583285 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20011ca0 con 0x7f0d2415fa60 2023-06-23T18:37:12.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.410+0000 7f0d2bb40700 1 -- 172.21.15.79:0/3586583285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d240adf30 con 0x7f0d2415fa60 2023-06-23T18:37:12.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.411+0000 7f0d2bb40700 1 -- 172.21.15.79:0/3586583285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2404e240 con 0x7f0d2415fa60 2023-06-23T18:37:12.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.411+0000 7f0d1a7fc700 1 -- 172.21.15.79:0/3586583285 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0d2001c460 con 0x7f0d2415fa60 2023-06-23T18:37:12.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.411+0000 7f0d1a7fc700 1 --2- 172.21.15.79:0/3586583285 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0d0806ace0 0x7f0d0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:12.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.411+0000 7f0d298dc700 1 --2- 172.21.15.79:0/3586583285 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0d0806ace0 0x7f0d0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:12.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.411+0000 7f0d1a7fc700 1 -- 172.21.15.79:0/3586583285 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d20089150 con 0x7f0d2415fa60 2023-06-23T18:37:12.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.412+0000 7f0d298dc700 1 --2- 172.21.15.79:0/3586583285 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0d0806ace0 0x7f0d0806d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0d14013040 tx=0x7f0d14008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:12.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.416+0000 7f0d1a7fc700 1 -- 172.21.15.79:0/3586583285 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0d20054200 con 0x7f0d2415fa60 2023-06-23T18:37:12.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.615+0000 7f0d2bb40700 1 -- 172.21.15.79:0/3586583285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0d2406a7b0 con 0x7f0d2415fa60 2023-06-23T18:37:12.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.616+0000 7f0d1a7fc700 1 -- 172.21.15.79:0/3586583285 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0d20057820 con 0x7f0d2415fa60 2023-06-23T18:37:12.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.618+0000 7f0d2bb40700 1 -- 172.21.15.79:0/3586583285 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0d0806ace0 msgr2=0x7f0d0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:12.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.618+0000 7f0d2bb40700 1 --2- 172.21.15.79:0/3586583285 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0d0806ace0 0x7f0d0806d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0d14013040 tx=0x7f0d14008070 comp rx=0 tx=0).stop 2023-06-23T18:37:12.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.618+0000 7f0d2bb40700 1 -- 172.21.15.79:0/3586583285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d2415fa60 msgr2=0x7f0d241f27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:12.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.618+0000 7f0d2bb40700 1 --2- 172.21.15.79:0/3586583285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d2415fa60 0x7f0d241f27f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000c4a0 tx=0x7f0d2000c7b0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.618+0000 7f0d2bb40700 1 -- 172.21.15.79:0/3586583285 shutdown_connections 2023-06-23T18:37:12.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.618+0000 7f0d2bb40700 1 --2- 172.21.15.79:0/3586583285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d241609f0 0x7f0d240c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.618+0000 7f0d2bb40700 1 --2- 172.21.15.79:0/3586583285 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f0d0806ace0 0x7f0d0806d190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.618+0000 7f0d2bb40700 1 --2- 172.21.15.79:0/3586583285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d240b17d0 0x7f0d240aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.618+0000 7f0d2bb40700 1 --2- 172.21.15.79:0/3586583285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d2415fa60 0x7f0d241f27f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:12.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.618+0000 7f0d2bb40700 1 -- 172.21.15.79:0/3586583285 >> 172.21.15.79:0/3586583285 conn(0x7f0d24089fc0 msgr2=0x7f0d240b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:12.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.618+0000 7f0d2bb40700 1 -- 172.21.15.79:0/3586583285 shutdown_connections 2023-06-23T18:37:12.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:12.618+0000 7f0d2bb40700 1 -- 172.21.15.79:0/3586583285 wait complete. 2023-06-23T18:37:12.625 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-23T18:37:12.634 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-06-23T18:37:12.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:12.636 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-06-23T18:37:12.677 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[108822]: fuse finished with error 0 and tester_r 0 2023-06-23T18:37:12.679 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:37:12.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:12.679 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-06-23T18:37:12.806 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:37:12.806 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:37:12.824 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=435 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-06-23T18:37:12.824 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:37:12.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:12.825 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-06-23T18:37:12.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:37:12.881 INFO:teuthology.orchestra.run.smithi079.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-06-23T18:37:12.881 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:37:12.881 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:37:12.900 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19111 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-06-23T18:37:12.900 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:37:12.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:12.901 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-06-23T18:37:12.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:37:12.959 INFO:teuthology.orchestra.run.smithi155.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-06-23T18:37:12.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:37:13.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.075+0000 7fbbfaed8700 1 Processor -- start 2023-06-23T18:37:13.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.075+0000 7fbbfaed8700 1 -- start start 2023-06-23T18:37:13.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.076+0000 7fbbfaed8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbbf415a380 0x7fbbf40ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.076+0000 7fbbfaed8700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbbf40cea30 0x7fbbf40b1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.076+0000 7fbbfaed8700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf40cee20 con 0x7fbbf415a750 2023-06-23T18:37:13.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.076+0000 7fbbfaed8700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf40b1d40 con 0x7fbbf415a380 2023-06-23T18:37:13.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.076+0000 7fbbfaed8700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf40b2020 con 0x7fbbf40cea30 2023-06-23T18:37:13.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.076+0000 7fbbf8c74700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbbf415a380 0x7fbbf40ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.076+0000 7fbbf8c74700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbbf415a380 0x7fbbf40ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41930/0 (socket says 172.21.15.79:41930) 2023-06-23T18:37:13.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.076+0000 7fbbf8c74700 1 -- 172.21.15.79:0/2249651943 learned_addr learned my addr 172.21.15.79:0/2249651943 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:13.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.076+0000 7fbbf8c74700 1 -- 172.21.15.79:0/2249651943 >> v2:172.21.15.155:3300/0 conn(0x7fbbf40cea30 msgr2=0x7fbbf40b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:13.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.076+0000 7fbbf8c74700 1 --2- 172.21.15.79:0/2249651943 >> v2:172.21.15.155:3300/0 conn(0x7fbbf40cea30 0x7fbbf40b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.076+0000 7fbbf8c74700 1 -- 172.21.15.79:0/2249651943 >> v1:172.21.15.155:6789/0 conn(0x7fbbf415a750 legacy=0x7fbbf40ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:13.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.076+0000 7fbbf8c74700 1 -- 172.21.15.79:0/2249651943 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf40ad650 con 0x7fbbf415a380 2023-06-23T18:37:13.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.076+0000 7fbbf8c74700 1 --2- 172.21.15.79:0/2249651943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbf415a380 0x7fbbf40ce4f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40c43f0 tx=0x7fbbe8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c14945925e0625 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:13.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.077+0000 7fbbf37fe700 1 -- 172.21.15.79:0/2249651943 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8005e30 con 0x7fbbf415a380 2023-06-23T18:37:13.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.077+0000 7fbbf37fe700 1 -- 172.21.15.79:0/2249651943 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe8006030 con 0x7fbbf415a380 2023-06-23T18:37:13.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.077+0000 7fbbf37fe700 1 -- 172.21.15.79:0/2249651943 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80063e0 con 0x7fbbf415a380 2023-06-23T18:37:13.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.077+0000 7fbbfaed8700 1 -- 172.21.15.79:0/2249651943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbf415a380 msgr2=0x7fbbf40ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.077+0000 7fbbfaed8700 1 --2- 172.21.15.79:0/2249651943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbf415a380 0x7fbbf40ce4f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40c43f0 tx=0x7fbbe8009bd0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.077+0000 7fbbfaed8700 1 -- 172.21.15.79:0/2249651943 shutdown_connections 2023-06-23T18:37:13.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.077+0000 7fbbfaed8700 1 --2- 172.21.15.79:0/2249651943 >> v2:172.21.15.155:3300/0 conn(0x7fbbf40cea30 0x7fbbf40b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.077+0000 7fbbfaed8700 1 --2- 172.21.15.79:0/2249651943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbf415a380 0x7fbbf40ce4f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.077+0000 7fbbfaed8700 1 -- 172.21.15.79:0/2249651943 >> 172.21.15.79:0/2249651943 conn(0x7fbbf4089fc0 msgr2=0x7fbbf4099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:13.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.077+0000 7fbbfaed8700 1 -- 172.21.15.79:0/2249651943 shutdown_connections 2023-06-23T18:37:13.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.077+0000 7fbbfaed8700 1 -- 172.21.15.79:0/2249651943 wait complete. 2023-06-23T18:37:13.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.078+0000 7fbbfaed8700 1 Processor -- start 2023-06-23T18:37:13.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.078+0000 7fbbfaed8700 1 -- start start 2023-06-23T18:37:13.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.079+0000 7fbbfaed8700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbf40cea30 0x7fbbf41f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.079+0000 7fbbfaed8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbf415a380 0x7fbbf415d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.079+0000 7fbbfaed8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbf415a750 0x7fbbf41591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.079+0000 7fbbfaed8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf40ccc50 con 0x7fbbf415a380 2023-06-23T18:37:13.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.079+0000 7fbbfaed8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf415c3c0 con 0x7fbbf415a750 2023-06-23T18:37:13.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.079+0000 7fbbfaed8700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf415c6a0 con 0x7fbbf40cea30 2023-06-23T18:37:13.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.079+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbf415a380 0x7fbbf415d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.079+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbf415a380 0x7fbbf415d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41938/0 (socket says 172.21.15.79:41938) 2023-06-23T18:37:13.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.079+0000 7fbbf3fff700 1 -- 172.21.15.79:0/1944630162 learned_addr learned my addr 172.21.15.79:0/1944630162 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:13.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.079+0000 7fbbf9475700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbf415a750 0x7fbbf41591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.080+0000 7fbbf8c74700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbf40cea30 0x7fbbf41f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.080+0000 7fbbf3fff700 1 -- 172.21.15.79:0/1944630162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbf40cea30 msgr2=0x7fbbf41f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.080+0000 7fbbf3fff700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbf40cea30 0x7fbbf41f1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.080+0000 7fbbf3fff700 1 -- 172.21.15.79:0/1944630162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbf415a750 msgr2=0x7fbbf41591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.080+0000 7fbbf3fff700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbf415a750 0x7fbbf41591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.080+0000 7fbbf3fff700 1 -- 172.21.15.79:0/1944630162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe8009500 con 0x7fbbf415a380 2023-06-23T18:37:13.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.080+0000 7fbbf8c74700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbf40cea30 0x7fbbf41f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:13.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.080+0000 7fbbf3fff700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbf415a380 0x7fbbf415d180 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000b410 tx=0x7fbbe000b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:13.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.080+0000 7fbbf17fa700 1 -- 172.21.15.79:0/1944630162 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0011830 con 0x7fbbf415a380 2023-06-23T18:37:13.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.080+0000 7fbbf17fa700 1 -- 172.21.15.79:0/1944630162 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe0011990 con 0x7fbbf415a380 2023-06-23T18:37:13.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.080+0000 7fbbf17fa700 1 -- 172.21.15.79:0/1944630162 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0011d40 con 0x7fbbf415a380 2023-06-23T18:37:13.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.080+0000 7fbbfaed8700 1 -- 172.21.15.79:0/1944630162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf415c980 con 0x7fbbf415a380 2023-06-23T18:37:13.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.080+0000 7fbbfaed8700 1 -- 172.21.15.79:0/1944630162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf41f2020 con 0x7fbbf415a380 2023-06-23T18:37:13.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.081+0000 7fbbfaed8700 1 -- 172.21.15.79:0/1944630162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf41f2240 con 0x7fbbf415a380 2023-06-23T18:37:13.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.081+0000 7fbbf17fa700 1 -- 172.21.15.79:0/1944630162 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbbe001a400 con 0x7fbbf415a380 2023-06-23T18:37:13.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.082+0000 7fbbf17fa700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbbd806ace0 0x7fbbd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.082+0000 7fbbf17fa700 1 -- 172.21.15.79:0/1944630162 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbbe0089350 con 0x7fbbf415a380 2023-06-23T18:37:13.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.082+0000 7fbbf8c74700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbbd806ace0 0x7fbbd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.082+0000 7fbbf8c74700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbbd806ace0 0x7fbbd806d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40c43f0 tx=0x7fbbe8008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:13.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.086+0000 7fbbf17fa700 1 -- 172.21.15.79:0/1944630162 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbbe0054400 con 0x7fbbf415a380 2023-06-23T18:37:13.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.314+0000 7fbbfaed8700 1 -- 172.21.15.79:0/1944630162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbf40677f0 con 0x7fbbf415a380 2023-06-23T18:37:13.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.315+0000 7fbbf17fa700 1 -- 172.21.15.79:0/1944630162 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4677 (secure 0 0 0) 0x7fbbe0057a20 con 0x7fbbf415a380 2023-06-23T18:37:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:13.320 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3101032943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":3101032943},{"type":"v1","addr":"172.21.15.79:6837","nonce":3101032943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2056827644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2056827644},{"type":"v1","addr":"172.21.15.155:6833","nonce":2056827644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1835571378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1835571378},{"type":"v1","addr":"172.21.15.155:6835","nonce":1835571378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:36:58.545517+0000","modified":"2023-06-23T18:37:01.557968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6836/1030786659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1030786659},{"type":"v1","addr":"172.21.15.79:6836","nonce":1030786659}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-23T18:37:13.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.317+0000 7fbbfaed8700 1 -- 172.21.15.79:0/1944630162 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbbd806ace0 msgr2=0x7fbbd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.317+0000 7fbbfaed8700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbbd806ace0 0x7fbbd806d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40c43f0 tx=0x7fbbe8008070 comp rx=0 tx=0).stop 2023-06-23T18:37:13.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.317+0000 7fbbfaed8700 1 -- 172.21.15.79:0/1944630162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbf415a380 msgr2=0x7fbbf415d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.317+0000 7fbbfaed8700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbf415a380 0x7fbbf415d180 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000b410 tx=0x7fbbe000b720 comp rx=0 tx=0).stop 2023-06-23T18:37:13.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.317+0000 7fbbfaed8700 1 -- 172.21.15.79:0/1944630162 shutdown_connections 2023-06-23T18:37:13.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.317+0000 7fbbfaed8700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fbbd806ace0 0x7fbbd806d190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.317+0000 7fbbfaed8700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbf415a750 0x7fbbf41591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.317+0000 7fbbfaed8700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbf40cea30 0x7fbbf41f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.317+0000 7fbbfaed8700 1 --2- 172.21.15.79:0/1944630162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbf415a380 0x7fbbf415d180 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.317+0000 7fbbfaed8700 1 -- 172.21.15.79:0/1944630162 >> 172.21.15.79:0/1944630162 conn(0x7fbbf4089fc0 msgr2=0x7fbbf406fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:13.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.317+0000 7fbbfaed8700 1 -- 172.21.15.79:0/1944630162 shutdown_connections 2023-06-23T18:37:13.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.317+0000 7fbbfaed8700 1 -- 172.21.15.79:0/1944630162 wait complete. 2023-06-23T18:37:13.324 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 16 2023-06-23T18:37:13.334 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:37:13.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.449+0000 7f59a8c64700 1 Processor -- start 2023-06-23T18:37:13.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.449+0000 7f59a8c64700 1 -- start start 2023-06-23T18:37:13.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a8c64700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f59a40b4f20 0x7f59a40c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a8c64700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f59a406a5a0 0x7f59a40cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a8c64700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a415a810 con 0x7f59a40b1f40 2023-06-23T18:37:13.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a8c64700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a40b52f0 con 0x7f59a406a5a0 2023-06-23T18:37:13.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a8c64700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a4071530 con 0x7f59a40b4f20 2023-06-23T18:37:13.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a2d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f59a406a5a0 0x7f59a40cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a2d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f59a406a5a0 0x7f59a40cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41948/0 (socket says 172.21.15.79:41948) 2023-06-23T18:37:13.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a2d9d700 1 -- 172.21.15.79:0/151110946 learned_addr learned my addr 172.21.15.79:0/151110946 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:13.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a2d9d700 1 -- 172.21.15.79:0/151110946 >> v2:172.21.15.155:3300/0 conn(0x7f59a40b4f20 msgr2=0x7f59a40c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:13.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a259c700 1 --2- 172.21.15.79:0/151110946 >> v2:172.21.15.155:3300/0 conn(0x7f59a40b4f20 0x7f59a40c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a2d9d700 1 --2- 172.21.15.79:0/151110946 >> v2:172.21.15.155:3300/0 conn(0x7f59a40b4f20 0x7f59a40c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a2d9d700 1 -- 172.21.15.79:0/151110946 >> v1:172.21.15.155:6789/0 conn(0x7f59a40b1f40 legacy=0x7f59a4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:13.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a2d9d700 1 -- 172.21.15.79:0/151110946 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a4071810 con 0x7f59a406a5a0 2023-06-23T18:37:13.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a2d9d700 1 --2- 172.21.15.79:0/151110946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59a406a5a0 0x7f59a40cb9a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f59a4071180 tx=0x7f599c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8f6de17cb2e4da server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:13.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a159a700 1 -- 172.21.15.79:0/151110946 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c00e070 con 0x7f59a406a5a0 2023-06-23T18:37:13.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a159a700 1 -- 172.21.15.79:0/151110946 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c003a20 con 0x7f59a406a5a0 2023-06-23T18:37:13.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.450+0000 7f59a159a700 1 -- 172.21.15.79:0/151110946 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c003dd0 con 0x7f59a406a5a0 2023-06-23T18:37:13.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.451+0000 7f59a8c64700 1 -- 172.21.15.79:0/151110946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59a406a5a0 msgr2=0x7f59a40cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.451+0000 7f59a8c64700 1 --2- 172.21.15.79:0/151110946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59a406a5a0 0x7f59a40cb9a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f59a4071180 tx=0x7f599c00b330 comp rx=0 tx=0).stop 2023-06-23T18:37:13.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.451+0000 7f59a8c64700 1 -- 172.21.15.79:0/151110946 shutdown_connections 2023-06-23T18:37:13.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.451+0000 7f59a8c64700 1 --2- 172.21.15.79:0/151110946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59a406a5a0 0x7f59a40cb9a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.451+0000 7f59a8c64700 1 --2- 172.21.15.79:0/151110946 >> v2:172.21.15.155:3300/0 conn(0x7f59a40b4f20 0x7f59a40c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.451+0000 7f59a8c64700 1 -- 172.21.15.79:0/151110946 >> 172.21.15.79:0/151110946 conn(0x7f59a406d660 msgr2=0x7f59a406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:13.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.451+0000 7f59a8c64700 1 -- 172.21.15.79:0/151110946 shutdown_connections 2023-06-23T18:37:13.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.451+0000 7f59a8c64700 1 -- 172.21.15.79:0/151110946 wait complete. 2023-06-23T18:37:13.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.452+0000 7f59a8c64700 1 Processor -- start 2023-06-23T18:37:13.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.452+0000 7f59a8c64700 1 -- start start 2023-06-23T18:37:13.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.453+0000 7f59a8c64700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59a406a5a0 0x7f59a4156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.453+0000 7f59a8c64700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59a40b1f40 0x7f59a4151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.453+0000 7f59a8c64700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a40b4f20 0x7f59a41f03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.453+0000 7f59a8c64700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a4156d60 con 0x7f59a406a5a0 2023-06-23T18:37:13.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.453+0000 7f59a8c64700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a41e17c0 con 0x7f59a40b4f20 2023-06-23T18:37:13.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.453+0000 7f59a8c64700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a4152fc0 con 0x7f59a40b1f40 2023-06-23T18:37:13.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.453+0000 7f59a259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59a406a5a0 0x7f59a4156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.453+0000 7f59a259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59a406a5a0 0x7f59a4156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41950/0 (socket says 172.21.15.79:41950) 2023-06-23T18:37:13.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.453+0000 7f59a259c700 1 -- 172.21.15.79:0/1038250468 learned_addr learned my addr 172.21.15.79:0/1038250468 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:13.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.453+0000 7f59a2d9d700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a40b4f20 0x7f59a41f03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.453+0000 7f59a1d9b700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59a40b1f40 0x7f59a4151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.453+0000 7f59a259c700 1 -- 172.21.15.79:0/1038250468 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59a40b1f40 msgr2=0x7f59a4151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.454+0000 7f59a259c700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59a40b1f40 0x7f59a4151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.454+0000 7f59a259c700 1 -- 172.21.15.79:0/1038250468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a40b4f20 msgr2=0x7f59a41f03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.454+0000 7f59a259c700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a40b4f20 0x7f59a41f03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.454+0000 7f59a259c700 1 -- 172.21.15.79:0/1038250468 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c009b30 con 0x7f59a406a5a0 2023-06-23T18:37:13.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.454+0000 7f59a2d9d700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a40b4f20 0x7f59a41f03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:13.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.454+0000 7f59a259c700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59a406a5a0 0x7f59a4156820 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f599400c7f0 tx=0x7f599400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:13.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.454+0000 7f5992ffd700 1 -- 172.21.15.79:0/1038250468 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994011800 con 0x7f59a406a5a0 2023-06-23T18:37:13.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.454+0000 7f5992ffd700 1 -- 172.21.15.79:0/1038250468 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5994011960 con 0x7f59a406a5a0 2023-06-23T18:37:13.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.454+0000 7f59a8c64700 1 -- 172.21.15.79:0/1038250468 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a4152510 con 0x7f59a406a5a0 2023-06-23T18:37:13.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.454+0000 7f5992ffd700 1 -- 172.21.15.79:0/1038250468 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994011cf0 con 0x7f59a406a5a0 2023-06-23T18:37:13.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.454+0000 7f59a8c64700 1 -- 172.21.15.79:0/1038250468 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a41e1170 con 0x7f59a406a5a0 2023-06-23T18:37:13.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.455+0000 7f59a8c64700 1 -- 172.21.15.79:0/1038250468 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a404e240 con 0x7f59a406a5a0 2023-06-23T18:37:13.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.455+0000 7f5992ffd700 1 -- 172.21.15.79:0/1038250468 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5994018360 con 0x7f59a406a5a0 2023-06-23T18:37:13.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.456+0000 7f5992ffd700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f598406ace0 0x7f598406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.456+0000 7f5992ffd700 1 -- 172.21.15.79:0/1038250468 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f59940893d0 con 0x7f59a406a5a0 2023-06-23T18:37:13.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.456+0000 7f59a1d9b700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f598406ace0 0x7f598406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.456+0000 7f59a1d9b700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f598406ace0 0x7f598406d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f59a40d0750 tx=0x7f5998009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:13.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.460+0000 7f5992ffd700 1 -- 172.21.15.79:0/1038250468 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5994054510 con 0x7f59a406a5a0 2023-06-23T18:37:13.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.649+0000 7f59a8c64700 1 -- 172.21.15.79:0/1038250468 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59a406e3d0 con 0x7f59a406a5a0 2023-06-23T18:37:13.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.650+0000 7f5992ffd700 1 -- 172.21.15.79:0/1038250468 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+18305 (secure 0 0 0) 0x7f5994057b30 con 0x7f59a406a5a0 2023-06-23T18:37:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:13.656 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":34,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:36:59.547297+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:36:56.964534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-23T18:36:57.507393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:37:13.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.652+0000 7f59a8c64700 1 -- 172.21.15.79:0/1038250468 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f598406ace0 msgr2=0x7f598406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.652+0000 7f59a8c64700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f598406ace0 0x7f598406d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f59a40d0750 tx=0x7f5998009040 comp rx=0 tx=0).stop 2023-06-23T18:37:13.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.652+0000 7f59a8c64700 1 -- 172.21.15.79:0/1038250468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59a406a5a0 msgr2=0x7f59a4156820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.652+0000 7f59a8c64700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59a406a5a0 0x7f59a4156820 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f599400c7f0 tx=0x7f599400cbb0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.652+0000 7f59a8c64700 1 -- 172.21.15.79:0/1038250468 shutdown_connections 2023-06-23T18:37:13.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.652+0000 7f59a8c64700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f598406ace0 0x7f598406d190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.652+0000 7f59a8c64700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a40b4f20 0x7f59a41f03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.652+0000 7f59a8c64700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59a406a5a0 0x7f59a4156820 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.652+0000 7f59a8c64700 1 --2- 172.21.15.79:0/1038250468 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59a40b1f40 0x7f59a4151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.652+0000 7f59a8c64700 1 -- 172.21.15.79:0/1038250468 >> 172.21.15.79:0/1038250468 conn(0x7f59a406d660 msgr2=0x7f59a40b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:13.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.652+0000 7f59a8c64700 1 -- 172.21.15.79:0/1038250468 shutdown_connections 2023-06-23T18:37:13.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.653+0000 7f59a8c64700 1 -- 172.21.15.79:0/1038250468 wait complete. 2023-06-23T18:37:13.671 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:35:46.467546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:35:46.735662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:36:56.964534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:36:57.507393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:37:13.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:37:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.792+0000 7f7c7aeac700 1 Processor -- start 2023-06-23T18:37:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.792+0000 7f7c7aeac700 1 -- start start 2023-06-23T18:37:13.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.793+0000 7f7c7aeac700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7c740cedf0 0x7f7c740b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.793+0000 7f7c7aeac700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c740b22c0 con 0x7f7c741438f0 2023-06-23T18:37:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.793+0000 7f7c7aeac700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c740b0b10 con 0x7f7c740d08c0 2023-06-23T18:37:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.793+0000 7f7c7aeac700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c740b0df0 con 0x7f7c740cedf0 2023-06-23T18:37:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.793+0000 7f7c78c48700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7c740cedf0 0x7f7c740b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.793+0000 7f7c73fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f7c741438f0 0x7f7c740b1bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45000/0 (socket says 172.21.15.79:45000) 2023-06-23T18:37:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.793+0000 7f7c73fff700 1 -- 172.21.15.79:0/4276083157 learned_addr learned my addr 172.21.15.79:0/4276083157 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:13.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.793+0000 7f7c78c48700 1 -- 172.21.15.79:0/4276083157 >> v1:172.21.15.155:6789/0 conn(0x7f7c740d08c0 legacy=0x7f7c740d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:13.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.793+0000 7f7c78c48700 1 -- 172.21.15.79:0/4276083157 >> v1:172.21.15.79:6789/0 conn(0x7f7c741438f0 legacy=0x7f7c740b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.793+0000 7f7c78c48700 1 -- 172.21.15.79:0/4276083157 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740b10d0 con 0x7f7c740cedf0 2023-06-23T18:37:13.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.794+0000 7f7c78c48700 1 --2- 172.21.15.79:0/4276083157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c740cedf0 0x7f7c740b02c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009820 tx=0x7f7c68009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22ccb91ed215228 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:13.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.794+0000 7f7c737fe700 1 -- 172.21.15.79:0/4276083157 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68005e30 con 0x7f7c740cedf0 2023-06-23T18:37:13.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.794+0000 7f7c737fe700 1 -- 172.21.15.79:0/4276083157 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c68004030 con 0x7f7c740cedf0 2023-06-23T18:37:13.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.794+0000 7f7c737fe700 1 -- 172.21.15.79:0/4276083157 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c680043c0 con 0x7f7c740cedf0 2023-06-23T18:37:13.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.794+0000 7f7c7aeac700 1 -- 172.21.15.79:0/4276083157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c740cedf0 msgr2=0x7f7c740b02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.794+0000 7f7c7aeac700 1 --2- 172.21.15.79:0/4276083157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c740cedf0 0x7f7c740b02c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009820 tx=0x7f7c68009b30 comp rx=0 tx=0).stop 2023-06-23T18:37:13.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.794+0000 7f7c7aeac700 1 -- 172.21.15.79:0/4276083157 shutdown_connections 2023-06-23T18:37:13.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.794+0000 7f7c7aeac700 1 --2- 172.21.15.79:0/4276083157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c740cedf0 0x7f7c740b02c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.794+0000 7f7c7aeac700 1 -- 172.21.15.79:0/4276083157 >> 172.21.15.79:0/4276083157 conn(0x7f7c7406a7b0 msgr2=0x7f7c74099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:13.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.794+0000 7f7c7aeac700 1 -- 172.21.15.79:0/4276083157 shutdown_connections 2023-06-23T18:37:13.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.794+0000 7f7c7aeac700 1 -- 172.21.15.79:0/4276083157 wait complete. 2023-06-23T18:37:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.795+0000 7f7c7aeac700 1 Processor -- start 2023-06-23T18:37:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.795+0000 7f7c7aeac700 1 -- start start 2023-06-23T18:37:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.796+0000 7f7c7aeac700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c740cedf0 0x7f7c7415c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.796+0000 7f7c7aeac700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c740d08c0 0x7f7c741f1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.796+0000 7f7c7aeac700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c741438f0 0x7f7c7415a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.796+0000 7f7c7aeac700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7406afa0 con 0x7f7c740d08c0 2023-06-23T18:37:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.796+0000 7f7c7aeac700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7406b560 con 0x7f7c741438f0 2023-06-23T18:37:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.796+0000 7f7c7aeac700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7415d0e0 con 0x7f7c740cedf0 2023-06-23T18:37:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.796+0000 7f7c73fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c740d08c0 0x7f7c741f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.796+0000 7f7c73fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c740d08c0 0x7f7c741f1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41966/0 (socket says 172.21.15.79:41966) 2023-06-23T18:37:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.796+0000 7f7c73fff700 1 -- 172.21.15.79:0/2815187618 learned_addr learned my addr 172.21.15.79:0/2815187618 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.797+0000 7f7c78c48700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c740cedf0 0x7f7c7415c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.797+0000 7f7c79449700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c741438f0 0x7f7c7415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.797+0000 7f7c73fff700 1 -- 172.21.15.79:0/2815187618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c740cedf0 msgr2=0x7f7c7415c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.797+0000 7f7c73fff700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c740cedf0 0x7f7c7415c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.797+0000 7f7c73fff700 1 -- 172.21.15.79:0/2815187618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c741438f0 msgr2=0x7f7c7415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.797+0000 7f7c73fff700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c741438f0 0x7f7c7415a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.797+0000 7f7c73fff700 1 -- 172.21.15.79:0/2815187618 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c68009430 con 0x7f7c740d08c0 2023-06-23T18:37:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.797+0000 7f7c78c48700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c740cedf0 0x7f7c7415c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.797+0000 7f7c73fff700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c740d08c0 0x7f7c741f1570 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000c4c0 tx=0x7f7c6000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.797+0000 7f7c717fa700 1 -- 172.21.15.79:0/2815187618 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c60005d70 con 0x7f7c740d08c0 2023-06-23T18:37:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.797+0000 7f7c717fa700 1 -- 172.21.15.79:0/2815187618 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c600070d0 con 0x7f7c740d08c0 2023-06-23T18:37:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.797+0000 7f7c7aeac700 1 -- 172.21.15.79:0/2815187618 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7415d3c0 con 0x7f7c740d08c0 2023-06-23T18:37:13.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.797+0000 7f7c717fa700 1 -- 172.21.15.79:0/2815187618 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c60007460 con 0x7f7c740d08c0 2023-06-23T18:37:13.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.797+0000 7f7c7aeac700 1 -- 172.21.15.79:0/2815187618 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7415ccd0 con 0x7f7c740d08c0 2023-06-23T18:37:13.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.798+0000 7f7c7aeac700 1 -- 172.21.15.79:0/2815187618 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c740672c0 con 0x7f7c740d08c0 2023-06-23T18:37:13.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.798+0000 7f7c717fa700 1 -- 172.21.15.79:0/2815187618 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7c600075c0 con 0x7f7c740d08c0 2023-06-23T18:37:13.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.799+0000 7f7c717fa700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7c5806ace0 0x7f7c5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:13.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.799+0000 7f7c717fa700 1 -- 172.21.15.79:0/2815187618 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7c60088920 con 0x7f7c740d08c0 2023-06-23T18:37:13.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.799+0000 7f7c78c48700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7c5806ace0 0x7f7c5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:13.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.799+0000 7f7c78c48700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7c5806ace0 0x7f7c5806d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7c68000c00 tx=0x7f7c68008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:13.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.803+0000 7f7c717fa700 1 -- 172.21.15.79:0/2815187618 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7c600539d0 con 0x7f7c740d08c0 2023-06-23T18:37:13.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.988+0000 7f7c7aeac700 1 -- 172.21.15.79:0/2815187618 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7c7406b6a0 con 0x7f7c740d08c0 2023-06-23T18:37:13.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.989+0000 7f7c717fa700 1 -- 172.21.15.79:0/2815187618 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7c600170a0 con 0x7f7c740d08c0 2023-06-23T18:37:13.994 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-23T18:37:13.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.991+0000 7f7c7aeac700 1 -- 172.21.15.79:0/2815187618 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7c5806ace0 msgr2=0x7f7c5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.991+0000 7f7c7aeac700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7c5806ace0 0x7f7c5806d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7c68000c00 tx=0x7f7c68008070 comp rx=0 tx=0).stop 2023-06-23T18:37:13.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.991+0000 7f7c7aeac700 1 -- 172.21.15.79:0/2815187618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c740d08c0 msgr2=0x7f7c741f1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:13.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.991+0000 7f7c7aeac700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c740d08c0 0x7f7c741f1570 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000c4c0 tx=0x7f7c6000c880 comp rx=0 tx=0).stop 2023-06-23T18:37:13.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.991+0000 7f7c7aeac700 1 -- 172.21.15.79:0/2815187618 shutdown_connections 2023-06-23T18:37:13.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.991+0000 7f7c7aeac700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f7c5806ace0 0x7f7c5806d190 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.991+0000 7f7c7aeac700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c741438f0 0x7f7c7415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.991+0000 7f7c7aeac700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c740cedf0 0x7f7c7415c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.991+0000 7f7c7aeac700 1 --2- 172.21.15.79:0/2815187618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c740d08c0 0x7f7c741f1570 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:13.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.991+0000 7f7c7aeac700 1 -- 172.21.15.79:0/2815187618 >> 172.21.15.79:0/2815187618 conn(0x7f7c7406a7b0 msgr2=0x7f7c740ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:13.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.991+0000 7f7c7aeac700 1 -- 172.21.15.79:0/2815187618 shutdown_connections 2023-06-23T18:37:13.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:13.991+0000 7f7c7aeac700 1 -- 172.21.15.79:0/2815187618 wait complete. 2023-06-23T18:37:14.008 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:37:14.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.126+0000 7f00cc3b9700 1 Processor -- start 2023-06-23T18:37:14.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.126+0000 7f00cc3b9700 1 -- start start 2023-06-23T18:37:14.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.127+0000 7f00cc3b9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00c40cd440 0x7f00c41608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.127+0000 7f00cc3b9700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c40cdc90 con 0x7f00c41569c0 2023-06-23T18:37:14.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.127+0000 7f00cc3b9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c415d0f0 con 0x7f00c406a5a0 2023-06-23T18:37:14.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.127+0000 7f00cc3b9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40cf570 con 0x7f00c40cd440 2023-06-23T18:37:14.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.127+0000 7f00ca155700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f00c41569c0 0x7f00c4069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45014/0 (socket says 172.21.15.79:45014) 2023-06-23T18:37:14.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.127+0000 7f00ca155700 1 -- 172.21.15.79:0/2173128443 learned_addr learned my addr 172.21.15.79:0/2173128443 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.127+0000 7f00ca956700 1 --2- 172.21.15.79:0/2173128443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00c40cd440 0x7f00c41608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.127+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040963375 0 0) 0x7f00b4001110 con 0x7f00c41569c0 2023-06-23T18:37:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.127+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00b00035b0 con 0x7f00c41569c0 2023-06-23T18:37:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933098083 0 0) 0x7f00c415d0f0 con 0x7f00c406a5a0 2023-06-23T18:37:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00b4001110 con 0x7f00c406a5a0 2023-06-23T18:37:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3504575121 0 0) 0x7f00b00035b0 con 0x7f00c41569c0 2023-06-23T18:37:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00c40cd440 msgr2=0x7f00c41608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 --2- 172.21.15.79:0/2173128443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00c40cd440 0x7f00c41608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 >> v1:172.21.15.155:6789/0 conn(0x7f00c406a5a0 legacy=0x7f00c40b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c4156d90 con 0x7f00c41569c0 2023-06-23T18:37:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f00b4001800 con 0x7f00c41569c0 2023-06-23T18:37:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 >> v1:172.21.15.79:6789/0 conn(0x7f00c41569c0 legacy=0x7f00c4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 --2- 172.21.15.79:0/2173128443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b0003bc0 0x7f00b0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 --2- 172.21.15.79:0/2173128443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00c40cd440 0x7f00b0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 --2- 172.21.15.79:0/2173128443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00b0004d90 0x7f00b0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b000bac0 con 0x7f00b0004d90 2023-06-23T18:37:14.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b000bc00 con 0x7f00b0003bc0 2023-06-23T18:37:14.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b000bd70 con 0x7f00c40cd440 2023-06-23T18:37:14.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00ca956700 1 --2- 172.21.15.79:0/2173128443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00b0004d90 0x7f00b0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9954700 1 --2- 172.21.15.79:0/2173128443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00c40cd440 0x7f00b0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00ca155700 1 --2- 172.21.15.79:0/2173128443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b0003bc0 0x7f00b0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00ca956700 1 -- 172.21.15.79:0/2173128443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00c40cd440 msgr2=0x7f00b0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00ca956700 1 --2- 172.21.15.79:0/2173128443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00c40cd440 0x7f00b0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00ca956700 1 -- 172.21.15.79:0/2173128443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b0003bc0 msgr2=0x7f00b0003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00ca956700 1 --2- 172.21.15.79:0/2173128443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b0003bc0 0x7f00b0003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00ca956700 1 -- 172.21.15.79:0/2173128443 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b00035b0 con 0x7f00b0004d90 2023-06-23T18:37:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00ca956700 1 --2- 172.21.15.79:0/2173128443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00b0004d90 0x7f00b0005200 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009f90 tx=0x7f00b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6745c2228fe4c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc00eb70 con 0x7f00b0004d90 2023-06-23T18:37:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.128+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc00f040 con 0x7f00b0004d90 2023-06-23T18:37:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.129+0000 7f00c9153700 1 -- 172.21.15.79:0/2173128443 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc014480 con 0x7f00b0004d90 2023-06-23T18:37:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.129+0000 7f00cc3b9700 1 -- 172.21.15.79:0/2173128443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00b0004d90 msgr2=0x7f00b0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.129+0000 7f00cc3b9700 1 --2- 172.21.15.79:0/2173128443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00b0004d90 0x7f00b0005200 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009f90 tx=0x7f00b0005880 comp rx=0 tx=0).stop 2023-06-23T18:37:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.129+0000 7f00ca956700 1 -- 172.21.15.79:0/2173128443 reap_dead start 2023-06-23T18:37:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.129+0000 7f00cc3b9700 1 -- 172.21.15.79:0/2173128443 shutdown_connections 2023-06-23T18:37:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.129+0000 7f00cc3b9700 1 -- 172.21.15.79:0/2173128443 >> 172.21.15.79:0/2173128443 conn(0x7f00c406afa0 msgr2=0x7f00c406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.129+0000 7f00cc3b9700 1 -- 172.21.15.79:0/2173128443 shutdown_connections 2023-06-23T18:37:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.129+0000 7f00cc3b9700 1 -- 172.21.15.79:0/2173128443 wait complete. 2023-06-23T18:37:14.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.130+0000 7f00cc3b9700 1 Processor -- start 2023-06-23T18:37:14.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.130+0000 7f00cc3b9700 1 -- start start 2023-06-23T18:37:14.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.131+0000 7f00cc3b9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b0004d90 0x7f00c41f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.131+0000 7f00cc3b9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00c41f1110 0x7f00c41f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.131+0000 7f00cc3b9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00c40c43f0 0x7f00c40c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.131+0000 7f00cc3b9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c41f6130 con 0x7f00c41f1110 2023-06-23T18:37:14.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.131+0000 7f00cc3b9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40c5c80 con 0x7f00b0004d90 2023-06-23T18:37:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.131+0000 7f00cc3b9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c41f1ac0 con 0x7f00c40c43f0 2023-06-23T18:37:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.131+0000 7f00c9954700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00c41f1110 0x7f00c41f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.131+0000 7f00c9954700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00c41f1110 0x7f00c41f1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41984/0 (socket says 172.21.15.79:41984) 2023-06-23T18:37:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.131+0000 7f00c9954700 1 -- 172.21.15.79:0/1788843698 learned_addr learned my addr 172.21.15.79:0/1788843698 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.131+0000 7f00ca155700 1 --2- 172.21.15.79:0/1788843698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b0004d90 0x7f00c41f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.131+0000 7f00ca956700 1 --2- 172.21.15.79:0/1788843698 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00c40c43f0 0x7f00c40c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.131+0000 7f00c9954700 1 -- 172.21.15.79:0/1788843698 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00c40c43f0 msgr2=0x7f00c40c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.131+0000 7f00c9954700 1 --2- 172.21.15.79:0/1788843698 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00c40c43f0 0x7f00c40c4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.131+0000 7f00c9954700 1 -- 172.21.15.79:0/1788843698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b0004d90 msgr2=0x7f00c41f54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.132+0000 7f00c9954700 1 --2- 172.21.15.79:0/1788843698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b0004d90 0x7f00c41f54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.132+0000 7f00c9954700 1 -- 172.21.15.79:0/1788843698 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c40c4db0 con 0x7f00c41f1110 2023-06-23T18:37:14.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.132+0000 7f00c9954700 1 --2- 172.21.15.79:0/1788843698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00c41f1110 0x7f00c41f1580 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f00c000f810 tx=0x7f00c000fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:14.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.132+0000 7f00baffd700 1 -- 172.21.15.79:0/1788843698 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c00052e0 con 0x7f00c41f1110 2023-06-23T18:37:14.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.132+0000 7f00baffd700 1 -- 172.21.15.79:0/1788843698 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c0005440 con 0x7f00c41f1110 2023-06-23T18:37:14.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.132+0000 7f00baffd700 1 -- 172.21.15.79:0/1788843698 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c00038c0 con 0x7f00c41f1110 2023-06-23T18:37:14.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.132+0000 7f00cc3b9700 1 -- 172.21.15.79:0/1788843698 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c40c5070 con 0x7f00c41f1110 2023-06-23T18:37:14.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.132+0000 7f00cc3b9700 1 -- 172.21.15.79:0/1788843698 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c40c5630 con 0x7f00c41f1110 2023-06-23T18:37:14.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.133+0000 7f00cc3b9700 1 -- 172.21.15.79:0/1788843698 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c40672c0 con 0x7f00c41f1110 2023-06-23T18:37:14.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.133+0000 7f00baffd700 1 -- 172.21.15.79:0/1788843698 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f00c0003a20 con 0x7f00c41f1110 2023-06-23T18:37:14.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.134+0000 7f00baffd700 1 --2- 172.21.15.79:0/1788843698 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f00a806ace0 0x7f00a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.134+0000 7f00baffd700 1 -- 172.21.15.79:0/1788843698 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f00c0089410 con 0x7f00c41f1110 2023-06-23T18:37:14.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.134+0000 7f00ca155700 1 --2- 172.21.15.79:0/1788843698 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f00a806ace0 0x7f00a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.134+0000 7f00ca155700 1 --2- 172.21.15.79:0/1788843698 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f00a806ace0 0x7f00a806d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f00b4001820 tx=0x7f00b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:14.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.138+0000 7f00baffd700 1 -- 172.21.15.79:0/1788843698 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f00c00544c0 con 0x7f00c41f1110 2023-06-23T18:37:14.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.327+0000 7f00cc3b9700 1 -- 172.21.15.79:0/1788843698 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00c40c5dc0 con 0x7f00c41f1110 2023-06-23T18:37:14.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.327+0000 7f00baffd700 1 -- 172.21.15.79:0/1788843698 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f00c0057ae0 con 0x7f00c41f1110 2023-06-23T18:37:14.332 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-23T18:37:14.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.329+0000 7f00cc3b9700 1 -- 172.21.15.79:0/1788843698 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f00a806ace0 msgr2=0x7f00a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.329+0000 7f00cc3b9700 1 --2- 172.21.15.79:0/1788843698 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f00a806ace0 0x7f00a806d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f00b4001820 tx=0x7f00b4009040 comp rx=0 tx=0).stop 2023-06-23T18:37:14.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.329+0000 7f00cc3b9700 1 -- 172.21.15.79:0/1788843698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00c41f1110 msgr2=0x7f00c41f1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.329+0000 7f00cc3b9700 1 --2- 172.21.15.79:0/1788843698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00c41f1110 0x7f00c41f1580 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f00c000f810 tx=0x7f00c000fbd0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.329+0000 7f00cc3b9700 1 -- 172.21.15.79:0/1788843698 shutdown_connections 2023-06-23T18:37:14.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.329+0000 7f00cc3b9700 1 --2- 172.21.15.79:0/1788843698 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00c40c43f0 0x7f00c40c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.330+0000 7f00cc3b9700 1 --2- 172.21.15.79:0/1788843698 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f00a806ace0 0x7f00a806d190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.330+0000 7f00cc3b9700 1 --2- 172.21.15.79:0/1788843698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b0004d90 0x7f00c41f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.330+0000 7f00cc3b9700 1 --2- 172.21.15.79:0/1788843698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00c41f1110 0x7f00c41f1580 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.330+0000 7f00cc3b9700 1 -- 172.21.15.79:0/1788843698 >> 172.21.15.79:0/1788843698 conn(0x7f00c406afa0 msgr2=0x7f00c415c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:14.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.330+0000 7f00cc3b9700 1 -- 172.21.15.79:0/1788843698 shutdown_connections 2023-06-23T18:37:14.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.330+0000 7f00cc3b9700 1 -- 172.21.15.79:0/1788843698 wait complete. 2023-06-23T18:37:14.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:37:14.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.465+0000 7f902fff8700 1 Processor -- start 2023-06-23T18:37:14.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.465+0000 7f902fff8700 1 -- start start 2023-06-23T18:37:14.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902fff8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f90281438f0 0x7f90280b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902fff8700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f90280d08c0 0x7f90280d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902fff8700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90280d1200 con 0x7f90280cedf0 2023-06-23T18:37:14.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902fff8700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90280b0b10 con 0x7f90281438f0 2023-06-23T18:37:14.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902fff8700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90280b0df0 con 0x7f90280d08c0 2023-06-23T18:37:14.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902d593700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f90281438f0 0x7f90280b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902dd94700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f90280cedf0 0x7f90280b02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45034/0 (socket says 172.21.15.79:45034) 2023-06-23T18:37:14.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902d593700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f90281438f0 0x7f90280b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41994/0 (socket says 172.21.15.79:41994) 2023-06-23T18:37:14.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902dd94700 1 -- 172.21.15.79:0/1220961041 learned_addr learned my addr 172.21.15.79:0/1220961041 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:14.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902e595700 1 --2- 172.21.15.79:0/1220961041 >> v2:172.21.15.155:3300/0 conn(0x7f90280d08c0 0x7f90280d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902d593700 1 -- 172.21.15.79:0/1220961041 >> v2:172.21.15.155:3300/0 conn(0x7f90280d08c0 msgr2=0x7f90280d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902d593700 1 --2- 172.21.15.79:0/1220961041 >> v2:172.21.15.155:3300/0 conn(0x7f90280d08c0 0x7f90280d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902d593700 1 -- 172.21.15.79:0/1220961041 >> v1:172.21.15.79:6789/0 conn(0x7f90280cedf0 legacy=0x7f90280b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902d593700 1 -- 172.21.15.79:0/1220961041 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90280b10d0 con 0x7f90281438f0 2023-06-23T18:37:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902e595700 1 --2- 172.21.15.79:0/1220961041 >> v2:172.21.15.155:3300/0 conn(0x7f90280d08c0 0x7f90280d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902d593700 1 --2- 172.21.15.79:0/1220961041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90281438f0 0x7f90280b1bb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f90280668b0 tx=0x7f9024009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695af5fb002c3fde server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902cd92700 1 -- 172.21.15.79:0/1220961041 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9024011830 con 0x7f90281438f0 2023-06-23T18:37:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902cd92700 1 -- 172.21.15.79:0/1220961041 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9024011990 con 0x7f90281438f0 2023-06-23T18:37:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.466+0000 7f902cd92700 1 -- 172.21.15.79:0/1220961041 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9024011d20 con 0x7f90281438f0 2023-06-23T18:37:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.467+0000 7f902fff8700 1 -- 172.21.15.79:0/1220961041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90281438f0 msgr2=0x7f90280b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.467+0000 7f902fff8700 1 --2- 172.21.15.79:0/1220961041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90281438f0 0x7f90280b1bb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f90280668b0 tx=0x7f9024009b20 comp rx=0 tx=0).stop 2023-06-23T18:37:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.467+0000 7f902fff8700 1 -- 172.21.15.79:0/1220961041 shutdown_connections 2023-06-23T18:37:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.467+0000 7f902fff8700 1 --2- 172.21.15.79:0/1220961041 >> v2:172.21.15.155:3300/0 conn(0x7f90280d08c0 0x7f90280d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.467+0000 7f902fff8700 1 --2- 172.21.15.79:0/1220961041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90281438f0 0x7f90280b1bb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.467+0000 7f902fff8700 1 -- 172.21.15.79:0/1220961041 >> 172.21.15.79:0/1220961041 conn(0x7f902806a7b0 msgr2=0x7f9028099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:14.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.467+0000 7f902fff8700 1 -- 172.21.15.79:0/1220961041 shutdown_connections 2023-06-23T18:37:14.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.467+0000 7f902fff8700 1 -- 172.21.15.79:0/1220961041 wait complete. 2023-06-23T18:37:14.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.468+0000 7f902fff8700 1 Processor -- start 2023-06-23T18:37:14.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.468+0000 7f902fff8700 1 -- start start 2023-06-23T18:37:14.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902fff8700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90280cedf0 0x7f9028153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902fff8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90280d08c0 0x7f90280b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902fff8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90281438f0 0x7f90281553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902fff8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90280c1bf0 con 0x7f90280d08c0 2023-06-23T18:37:14.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902fff8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9028154e90 con 0x7f90281438f0 2023-06-23T18:37:14.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902fff8700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90280b3e80 con 0x7f90280cedf0 2023-06-23T18:37:14.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902d593700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90280d08c0 0x7f90280b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902d593700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90280d08c0 0x7f90280b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42000/0 (socket says 172.21.15.79:42000) 2023-06-23T18:37:14.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902d593700 1 -- 172.21.15.79:0/4282858855 learned_addr learned my addr 172.21.15.79:0/4282858855 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:14.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902d593700 1 -- 172.21.15.79:0/4282858855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90280cedf0 msgr2=0x7f9028153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902d593700 1 --2- 172.21.15.79:0/4282858855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90280cedf0 0x7f9028153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902d593700 1 -- 172.21.15.79:0/4282858855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90281438f0 msgr2=0x7f90281553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902d593700 1 --2- 172.21.15.79:0/4282858855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90281438f0 0x7f90281553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902d593700 1 -- 172.21.15.79:0/4282858855 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90240094a0 con 0x7f90280d08c0 2023-06-23T18:37:14.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.469+0000 7f902d593700 1 --2- 172.21.15.79:0/4282858855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90280d08c0 0x7f90280b3940 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f90240090b0 tx=0x7f9024004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:14.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.470+0000 7f901e7fc700 1 -- 172.21.15.79:0/4282858855 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90240045f0 con 0x7f90280d08c0 2023-06-23T18:37:14.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.470+0000 7f901e7fc700 1 -- 172.21.15.79:0/4282858855 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902401f070 con 0x7f90280d08c0 2023-06-23T18:37:14.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.470+0000 7f902fff8700 1 -- 172.21.15.79:0/4282858855 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90280b4090 con 0x7f90280d08c0 2023-06-23T18:37:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.470+0000 7f901e7fc700 1 -- 172.21.15.79:0/4282858855 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9024003b00 con 0x7f90280d08c0 2023-06-23T18:37:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.470+0000 7f902fff8700 1 -- 172.21.15.79:0/4282858855 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90280c1810 con 0x7f90280d08c0 2023-06-23T18:37:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.470+0000 7f902fff8700 1 -- 172.21.15.79:0/4282858855 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90280c1d30 con 0x7f90280d08c0 2023-06-23T18:37:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.471+0000 7f901e7fc700 1 -- 172.21.15.79:0/4282858855 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9024003c60 con 0x7f90280d08c0 2023-06-23T18:37:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.471+0000 7f901e7fc700 1 --2- 172.21.15.79:0/4282858855 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f900c06ace0 0x7f900c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.471+0000 7f901e7fc700 1 -- 172.21.15.79:0/4282858855 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9024021070 con 0x7f90280d08c0 2023-06-23T18:37:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.471+0000 7f902dd94700 1 --2- 172.21.15.79:0/4282858855 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f900c06ace0 0x7f900c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.472+0000 7f902dd94700 1 --2- 172.21.15.79:0/4282858855 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f900c06ace0 0x7f900c06d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9018007fd0 tx=0x7f9018007940 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:14.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.476+0000 7f901e7fc700 1 -- 172.21.15.79:0/4282858855 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f90280c1d30 con 0x7f90280d08c0 2023-06-23T18:37:14.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.665+0000 7f902fff8700 1 -- 172.21.15.79:0/4282858855 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90280c1d30 con 0x7f90280d08c0 2023-06-23T18:37:14.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.665+0000 7f901e7fc700 1 -- 172.21.15.79:0/4282858855 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9024015020 con 0x7f90280d08c0 2023-06-23T18:37:14.671 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-23T18:37:14.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.667+0000 7f902fff8700 1 -- 172.21.15.79:0/4282858855 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f900c06ace0 msgr2=0x7f900c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.667+0000 7f902fff8700 1 --2- 172.21.15.79:0/4282858855 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f900c06ace0 0x7f900c06d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9018007fd0 tx=0x7f9018007940 comp rx=0 tx=0).stop 2023-06-23T18:37:14.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.667+0000 7f902fff8700 1 -- 172.21.15.79:0/4282858855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90280d08c0 msgr2=0x7f90280b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.667+0000 7f902fff8700 1 --2- 172.21.15.79:0/4282858855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90280d08c0 0x7f90280b3940 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f90240090b0 tx=0x7f9024004750 comp rx=0 tx=0).stop 2023-06-23T18:37:14.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.667+0000 7f902fff8700 1 -- 172.21.15.79:0/4282858855 shutdown_connections 2023-06-23T18:37:14.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.667+0000 7f902fff8700 1 --2- 172.21.15.79:0/4282858855 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f900c06ace0 0x7f900c06d190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.667+0000 7f902fff8700 1 --2- 172.21.15.79:0/4282858855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90281438f0 0x7f90281553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.667+0000 7f902fff8700 1 --2- 172.21.15.79:0/4282858855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90280cedf0 0x7f9028153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.667+0000 7f902fff8700 1 --2- 172.21.15.79:0/4282858855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90280d08c0 0x7f90280b3940 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.667+0000 7f902fff8700 1 -- 172.21.15.79:0/4282858855 >> 172.21.15.79:0/4282858855 conn(0x7f902806a7b0 msgr2=0x7f90280ad530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:14.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.667+0000 7f902fff8700 1 -- 172.21.15.79:0/4282858855 shutdown_connections 2023-06-23T18:37:14.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.667+0000 7f902fff8700 1 -- 172.21.15.79:0/4282858855 wait complete. 2023-06-23T18:37:14.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:37:14.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.804+0000 7f68bfa8b700 1 Processor -- start 2023-06-23T18:37:14.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.804+0000 7f68bfa8b700 1 -- start start 2023-06-23T18:37:14.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68bfa8b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68b815d750 0x7f68b815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68bfa8b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68b806a5a0 0x7f68b80b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68bfa8b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b8071cf0 con 0x7f68b8072180 2023-06-23T18:37:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68bfa8b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b815ecf0 con 0x7f68b806a5a0 2023-06-23T18:37:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68bfa8b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b815db20 con 0x7f68b815d750 2023-06-23T18:37:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68be028700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68b806a5a0 0x7f68b80b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68be028700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68b806a5a0 0x7f68b80b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42010/0 (socket says 172.21.15.79:42010) 2023-06-23T18:37:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68be028700 1 -- 172.21.15.79:0/1557963697 learned_addr learned my addr 172.21.15.79:0/1557963697 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68bd827700 1 --2- 172.21.15.79:0/1557963697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68b815d750 0x7f68b815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68be028700 1 -- 172.21.15.79:0/1557963697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68b815d750 msgr2=0x7f68b815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:14.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68be028700 1 --2- 172.21.15.79:0/1557963697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68b815d750 0x7f68b815e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68be028700 1 -- 172.21.15.79:0/1557963697 >> v1:172.21.15.155:6789/0 conn(0x7f68b8072180 legacy=0x7f68b8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68be028700 1 -- 172.21.15.79:0/1557963697 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b80ca310 con 0x7f68b806a5a0 2023-06-23T18:37:14.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68bc825700 1 -- 172.21.15.79:0/1557963697 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2556282788 0 0) 0x7f68b8071cf0 con 0x7f68b8072180 2023-06-23T18:37:14.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68be028700 1 --2- 172.21.15.79:0/1557963697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68b806a5a0 0x7f68b80b12f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009e50 tx=0x7f68b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49df1cd686788e5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:14.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.805+0000 7f68bc825700 1 -- 172.21.15.79:0/1557963697 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b000e070 con 0x7f68b806a5a0 2023-06-23T18:37:14.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.806+0000 7f68bc825700 1 -- 172.21.15.79:0/1557963697 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b000b910 con 0x7f68b806a5a0 2023-06-23T18:37:14.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.806+0000 7f68bc825700 1 -- 172.21.15.79:0/1557963697 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b000bcc0 con 0x7f68b806a5a0 2023-06-23T18:37:14.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.806+0000 7f68bfa8b700 1 -- 172.21.15.79:0/1557963697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68b806a5a0 msgr2=0x7f68b80b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.806+0000 7f68bfa8b700 1 --2- 172.21.15.79:0/1557963697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68b806a5a0 0x7f68b80b12f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009e50 tx=0x7f68b000b330 comp rx=0 tx=0).stop 2023-06-23T18:37:14.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.806+0000 7f68bfa8b700 1 -- 172.21.15.79:0/1557963697 shutdown_connections 2023-06-23T18:37:14.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.806+0000 7f68bfa8b700 1 --2- 172.21.15.79:0/1557963697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68b806a5a0 0x7f68b80b12f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.806+0000 7f68bfa8b700 1 --2- 172.21.15.79:0/1557963697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68b815d750 0x7f68b815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.806+0000 7f68bfa8b700 1 -- 172.21.15.79:0/1557963697 >> 172.21.15.79:0/1557963697 conn(0x7f68b806afa0 msgr2=0x7f68b806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.806+0000 7f68bfa8b700 1 -- 172.21.15.79:0/1557963697 shutdown_connections 2023-06-23T18:37:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.806+0000 7f68bfa8b700 1 -- 172.21.15.79:0/1557963697 wait complete. 2023-06-23T18:37:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.807+0000 7f68bfa8b700 1 Processor -- start 2023-06-23T18:37:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.807+0000 7f68bfa8b700 1 -- start start 2023-06-23T18:37:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.808+0000 7f68bfa8b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68b806a5a0 0x7f68b81ef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.808+0000 7f68bfa8b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68b8072180 0x7f68b8155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.808+0000 7f68bfa8b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68b815d750 0x7f68b8156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.808+0000 7f68bfa8b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b80affc0 con 0x7f68b815d750 2023-06-23T18:37:14.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.808+0000 7f68bfa8b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b81f0740 con 0x7f68b806a5a0 2023-06-23T18:37:14.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.808+0000 7f68bfa8b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b81efea0 con 0x7f68b8072180 2023-06-23T18:37:14.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.808+0000 7f68be028700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68b815d750 0x7f68b8156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.808+0000 7f68be028700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68b815d750 0x7f68b8156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42016/0 (socket says 172.21.15.79:42016) 2023-06-23T18:37:14.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.808+0000 7f68be028700 1 -- 172.21.15.79:0/3712225012 learned_addr learned my addr 172.21.15.79:0/3712225012 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:14.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.808+0000 7f68bd827700 1 --2- 172.21.15.79:0/3712225012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68b806a5a0 0x7f68b81ef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.808+0000 7f68bd026700 1 --2- 172.21.15.79:0/3712225012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68b8072180 0x7f68b8155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.809+0000 7f68be028700 1 -- 172.21.15.79:0/3712225012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68b8072180 msgr2=0x7f68b8155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.809+0000 7f68be028700 1 --2- 172.21.15.79:0/3712225012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68b8072180 0x7f68b8155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.809+0000 7f68be028700 1 -- 172.21.15.79:0/3712225012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68b806a5a0 msgr2=0x7f68b81ef7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:14.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.809+0000 7f68be028700 1 --2- 172.21.15.79:0/3712225012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68b806a5a0 0x7f68b81ef7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:14.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.809+0000 7f68be028700 1 -- 172.21.15.79:0/3712225012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b0009b30 con 0x7f68b815d750 2023-06-23T18:37:14.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.809+0000 7f68be028700 1 --2- 172.21.15.79:0/3712225012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68b815d750 0x7f68b8156530 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f68b80d0970 tx=0x7f68b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:14.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.809+0000 7f68ae7fc700 1 -- 172.21.15.79:0/3712225012 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b0004880 con 0x7f68b815d750 2023-06-23T18:37:14.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.809+0000 7f68ae7fc700 1 -- 172.21.15.79:0/3712225012 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b00099b0 con 0x7f68b815d750 2023-06-23T18:37:14.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.809+0000 7f68bfa8b700 1 -- 172.21.15.79:0/3712225012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b81f0180 con 0x7f68b815d750 2023-06-23T18:37:14.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.809+0000 7f68ae7fc700 1 -- 172.21.15.79:0/3712225012 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b001b670 con 0x7f68b815d750 2023-06-23T18:37:14.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.809+0000 7f68bfa8b700 1 -- 172.21.15.79:0/3712225012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b815a5a0 con 0x7f68b815d750 2023-06-23T18:37:14.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.810+0000 7f68bfa8b700 1 -- 172.21.15.79:0/3712225012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b80672c0 con 0x7f68b815d750 2023-06-23T18:37:14.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.810+0000 7f68ae7fc700 1 -- 172.21.15.79:0/3712225012 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f68b001b7d0 con 0x7f68b815d750 2023-06-23T18:37:14.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.810+0000 7f68ae7fc700 1 --2- 172.21.15.79:0/3712225012 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f689c06ace0 0x7f689c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:14.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.811+0000 7f68ae7fc700 1 -- 172.21.15.79:0/3712225012 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f68b008aba0 con 0x7f68b815d750 2023-06-23T18:37:14.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.811+0000 7f68bd827700 1 --2- 172.21.15.79:0/3712225012 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f689c06ace0 0x7f689c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:14.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.811+0000 7f68bd827700 1 --2- 172.21.15.79:0/3712225012 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f689c06ace0 0x7f689c06d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f68b815e270 tx=0x7f68b4009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:14.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:14.815+0000 7f68ae7fc700 1 -- 172.21.15.79:0/3712225012 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f68b0055ce0 con 0x7f68b815d750 2023-06-23T18:37:15.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.000+0000 7f68bfa8b700 1 -- 172.21.15.79:0/3712225012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f68b80cbcd0 con 0x7f68b815d750 2023-06-23T18:37:15.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.001+0000 7f68ae7fc700 1 -- 172.21.15.79:0/3712225012 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f68b0059300 con 0x7f68b815d750 2023-06-23T18:37:15.006 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-23T18:37:15.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.003+0000 7f68bfa8b700 1 -- 172.21.15.79:0/3712225012 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f689c06ace0 msgr2=0x7f689c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.003+0000 7f68bfa8b700 1 --2- 172.21.15.79:0/3712225012 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f689c06ace0 0x7f689c06d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f68b815e270 tx=0x7f68b4009210 comp rx=0 tx=0).stop 2023-06-23T18:37:15.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.003+0000 7f68bfa8b700 1 -- 172.21.15.79:0/3712225012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68b815d750 msgr2=0x7f68b8156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.003+0000 7f68bfa8b700 1 --2- 172.21.15.79:0/3712225012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68b815d750 0x7f68b8156530 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f68b80d0970 tx=0x7f68b0004ad0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.003+0000 7f68bfa8b700 1 -- 172.21.15.79:0/3712225012 shutdown_connections 2023-06-23T18:37:15.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.003+0000 7f68bfa8b700 1 --2- 172.21.15.79:0/3712225012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68b815d750 0x7f68b8156530 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.003+0000 7f68bfa8b700 1 --2- 172.21.15.79:0/3712225012 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f689c06ace0 0x7f689c06d190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.003+0000 7f68bfa8b700 1 --2- 172.21.15.79:0/3712225012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68b806a5a0 0x7f68b81ef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.003+0000 7f68bfa8b700 1 --2- 172.21.15.79:0/3712225012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68b8072180 0x7f68b8155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.003+0000 7f68bfa8b700 1 -- 172.21.15.79:0/3712225012 >> 172.21.15.79:0/3712225012 conn(0x7f68b806afa0 msgr2=0x7f68b815b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:15.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.003+0000 7f68bfa8b700 1 -- 172.21.15.79:0/3712225012 shutdown_connections 2023-06-23T18:37:15.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.003+0000 7f68bfa8b700 1 -- 172.21.15.79:0/3712225012 wait complete. 2023-06-23T18:37:15.020 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:37:15.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.136+0000 7fe5856fc700 1 Processor -- start 2023-06-23T18:37:15.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.136+0000 7fe5856fc700 1 -- start start 2023-06-23T18:37:15.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.136+0000 7fe5856fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe58015c0e0 0x7fe5800ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.136+0000 7fe5856fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe5801608f0 0x7fe5800c9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.136+0000 7fe5856fc700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5800cd240 con 0x7fe58015c4b0 2023-06-23T18:37:15.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.136+0000 7fe5856fc700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe580160e40 con 0x7fe5801608f0 2023-06-23T18:37:15.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.136+0000 7fe5856fc700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5800cccc0 con 0x7fe58015c0e0 2023-06-23T18:37:15.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57e7fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe5801608f0 0x7fe5800c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57e7fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe5801608f0 0x7fe5800c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42024/0 (socket says 172.21.15.79:42024) 2023-06-23T18:37:15.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57e7fc700 1 -- 172.21.15.79:0/1575786263 learned_addr learned my addr 172.21.15.79:0/1575786263 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:15.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57f7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe58015c4b0 0x7fe580070b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45064/0 (socket says 172.21.15.79:45064) 2023-06-23T18:37:15.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57effd700 1 --2- 172.21.15.79:0/1575786263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe58015c0e0 0x7fe5800ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57e7fc700 1 -- 172.21.15.79:0/1575786263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe58015c0e0 msgr2=0x7fe5800ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:15.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57e7fc700 1 --2- 172.21.15.79:0/1575786263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe58015c0e0 0x7fe5800ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57e7fc700 1 -- 172.21.15.79:0/1575786263 >> v1:172.21.15.79:6789/0 conn(0x7fe58015c4b0 legacy=0x7fe580070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57dffb700 1 -- 172.21.15.79:0/1575786263 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338675362 0 0) 0x7fe5800cd240 con 0x7fe58015c4b0 2023-06-23T18:37:15.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57e7fc700 1 -- 172.21.15.79:0/1575786263 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5800ccf60 con 0x7fe5801608f0 2023-06-23T18:37:15.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57effd700 1 --2- 172.21.15.79:0/1575786263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe58015c0e0 0x7fe5800ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:15.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57e7fc700 1 --2- 172.21.15.79:0/1575786263 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5801608f0 0x7fe5800c9610 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe5740098f0 tx=0x7fe574009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76aa35db66c4a5ea server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:15.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57dffb700 1 -- 172.21.15.79:0/1575786263 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57400b8a0 con 0x7fe5801608f0 2023-06-23T18:37:15.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57dffb700 1 -- 172.21.15.79:0/1575786263 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57400ba00 con 0x7fe5801608f0 2023-06-23T18:37:15.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.137+0000 7fe57dffb700 1 -- 172.21.15.79:0/1575786263 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57400bd90 con 0x7fe5801608f0 2023-06-23T18:37:15.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.138+0000 7fe5856fc700 1 -- 172.21.15.79:0/1575786263 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5801608f0 msgr2=0x7fe5800c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.138+0000 7fe5856fc700 1 --2- 172.21.15.79:0/1575786263 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5801608f0 0x7fe5800c9610 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe5740098f0 tx=0x7fe574009c00 comp rx=0 tx=0).stop 2023-06-23T18:37:15.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.138+0000 7fe5856fc700 1 -- 172.21.15.79:0/1575786263 shutdown_connections 2023-06-23T18:37:15.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.138+0000 7fe5856fc700 1 --2- 172.21.15.79:0/1575786263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe58015c0e0 0x7fe5800ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.138+0000 7fe5856fc700 1 --2- 172.21.15.79:0/1575786263 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5801608f0 0x7fe5800c9610 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.138+0000 7fe5856fc700 1 -- 172.21.15.79:0/1575786263 >> 172.21.15.79:0/1575786263 conn(0x7fe5800cfc50 msgr2=0x7fe5800b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:15.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.138+0000 7fe5856fc700 1 -- 172.21.15.79:0/1575786263 shutdown_connections 2023-06-23T18:37:15.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.138+0000 7fe5856fc700 1 -- 172.21.15.79:0/1575786263 wait complete. 2023-06-23T18:37:15.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.139+0000 7fe5856fc700 1 Processor -- start 2023-06-23T18:37:15.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.140+0000 7fe5856fc700 1 -- start start 2023-06-23T18:37:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.140+0000 7fe5856fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe58015c0e0 0x7fe5800aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.140+0000 7fe5856fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe58015c4b0 0x7fe5800abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.140+0000 7fe5856fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5801608f0 0x7fe5801f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.140+0000 7fe5856fc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5800af3d0 con 0x7fe58015c0e0 2023-06-23T18:37:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.140+0000 7fe5856fc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5800ac0a0 con 0x7fe58015c4b0 2023-06-23T18:37:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.140+0000 7fe5856fc700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5800af5e0 con 0x7fe5801608f0 2023-06-23T18:37:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe57effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe58015c0e0 0x7fe5800aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe57effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe58015c0e0 0x7fe5800aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42026/0 (socket says 172.21.15.79:42026) 2023-06-23T18:37:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe57effd700 1 -- 172.21.15.79:0/3204591538 learned_addr learned my addr 172.21.15.79:0/3204591538 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe57f7fe700 1 --2- 172.21.15.79:0/3204591538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5801608f0 0x7fe5801f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe57effd700 1 -- 172.21.15.79:0/3204591538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5801608f0 msgr2=0x7fe5801f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe57effd700 1 --2- 172.21.15.79:0/3204591538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5801608f0 0x7fe5801f14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe57effd700 1 -- 172.21.15.79:0/3204591538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe58015c4b0 msgr2=0x7fe5800abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe57e7fc700 1 --2- 172.21.15.79:0/3204591538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe58015c4b0 0x7fe5800abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe57effd700 1 --2- 172.21.15.79:0/3204591538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe58015c4b0 0x7fe5800abb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe57effd700 1 -- 172.21.15.79:0/3204591538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574009550 con 0x7fe58015c0e0 2023-06-23T18:37:15.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe57effd700 1 --2- 172.21.15.79:0/3204591538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe58015c0e0 0x7fe5800aee90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe57000e790 tx=0x7fe57000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:15.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe5677fe700 1 -- 172.21.15.79:0/3204591538 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570013610 con 0x7fe58015c0e0 2023-06-23T18:37:15.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe5677fe700 1 -- 172.21.15.79:0/3204591538 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe570013770 con 0x7fe58015c0e0 2023-06-23T18:37:15.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe5856fc700 1 -- 172.21.15.79:0/3204591538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5800af8c0 con 0x7fe58015c0e0 2023-06-23T18:37:15.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe5856fc700 1 -- 172.21.15.79:0/3204591538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5801f2200 con 0x7fe58015c0e0 2023-06-23T18:37:15.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.141+0000 7fe5677fe700 1 -- 172.21.15.79:0/3204591538 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570013b20 con 0x7fe58015c0e0 2023-06-23T18:37:15.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.142+0000 7fe5856fc700 1 -- 172.21.15.79:0/3204591538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5600052f0 con 0x7fe58015c0e0 2023-06-23T18:37:15.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.143+0000 7fe5677fe700 1 -- 172.21.15.79:0/3204591538 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe570013c80 con 0x7fe58015c0e0 2023-06-23T18:37:15.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.143+0000 7fe5677fe700 1 --2- 172.21.15.79:0/3204591538 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe56c06ace0 0x7fe56c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.143+0000 7fe57e7fc700 1 --2- 172.21.15.79:0/3204591538 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe56c06ace0 0x7fe56c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.143+0000 7fe5677fe700 1 -- 172.21.15.79:0/3204591538 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe570089080 con 0x7fe58015c0e0 2023-06-23T18:37:15.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.144+0000 7fe57e7fc700 1 --2- 172.21.15.79:0/3204591538 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe56c06ace0 0x7fe56c06d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe58015fbd0 tx=0x7fe574008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:15.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.147+0000 7fe5677fe700 1 -- 172.21.15.79:0/3204591538 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe570057bc0 con 0x7fe58015c0e0 2023-06-23T18:37:15.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.377+0000 7fe5856fc700 1 -- 172.21.15.79:0/3204591538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5600055a0 con 0x7fe58015c0e0 2023-06-23T18:37:15.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.378+0000 7fe5677fe700 1 -- 172.21.15.79:0/3204591538 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4677 (secure 0 0 0) 0x7fe570057750 con 0x7fe58015c0e0 2023-06-23T18:37:15.383 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:15.383 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3101032943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":3101032943},{"type":"v1","addr":"172.21.15.79:6837","nonce":3101032943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2056827644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2056827644},{"type":"v1","addr":"172.21.15.155:6833","nonce":2056827644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1835571378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1835571378},{"type":"v1","addr":"172.21.15.155:6835","nonce":1835571378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:36:58.545517+0000","modified":"2023-06-23T18:37:01.557968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6836/1030786659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1030786659},{"type":"v1","addr":"172.21.15.79:6836","nonce":1030786659}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-23T18:37:15.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.380+0000 7fe5856fc700 1 -- 172.21.15.79:0/3204591538 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe56c06ace0 msgr2=0x7fe56c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.380+0000 7fe5856fc700 1 --2- 172.21.15.79:0/3204591538 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe56c06ace0 0x7fe56c06d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe58015fbd0 tx=0x7fe574008030 comp rx=0 tx=0).stop 2023-06-23T18:37:15.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.380+0000 7fe5856fc700 1 -- 172.21.15.79:0/3204591538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe58015c0e0 msgr2=0x7fe5800aee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.380+0000 7fe5856fc700 1 --2- 172.21.15.79:0/3204591538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe58015c0e0 0x7fe5800aee90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe57000e790 tx=0x7fe57000eb50 comp rx=0 tx=0).stop 2023-06-23T18:37:15.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.380+0000 7fe5856fc700 1 -- 172.21.15.79:0/3204591538 shutdown_connections 2023-06-23T18:37:15.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.380+0000 7fe5856fc700 1 --2- 172.21.15.79:0/3204591538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5801608f0 0x7fe5801f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.380+0000 7fe5856fc700 1 --2- 172.21.15.79:0/3204591538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe58015c0e0 0x7fe5800aee90 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.380+0000 7fe5856fc700 1 --2- 172.21.15.79:0/3204591538 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe56c06ace0 0x7fe56c06d190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.380+0000 7fe5856fc700 1 --2- 172.21.15.79:0/3204591538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe58015c4b0 0x7fe5800abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.380+0000 7fe5856fc700 1 -- 172.21.15.79:0/3204591538 >> 172.21.15.79:0/3204591538 conn(0x7fe5800cfc50 msgr2=0x7fe580069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:15.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.380+0000 7fe5856fc700 1 -- 172.21.15.79:0/3204591538 shutdown_connections 2023-06-23T18:37:15.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.380+0000 7fe5856fc700 1 -- 172.21.15.79:0/3204591538 wait complete. 2023-06-23T18:37:15.387 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 16 2023-06-23T18:37:15.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:37:15.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.511+0000 7f2b30ca9700 1 Processor -- start 2023-06-23T18:37:15.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.512+0000 7f2b30ca9700 1 -- start start 2023-06-23T18:37:15.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.512+0000 7f2b30ca9700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2b2c0b1270 0x7f2b2c15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.512+0000 7f2b30ca9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2c0aee30 0x7f2b2c15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.512+0000 7f2b30ca9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2c1608f0 con 0x7f2b2c071490 2023-06-23T18:37:15.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.512+0000 7f2b30ca9700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c160e20 con 0x7f2b2c0b1270 2023-06-23T18:37:15.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.512+0000 7f2b30ca9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0cd4f0 con 0x7f2b2c0aee30 2023-06-23T18:37:15.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.513+0000 7f2b29d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2b2c0b1270 0x7f2b2c15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.513+0000 7f2b29d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2b2c0b1270 0x7f2b2c15c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42034/0 (socket says 172.21.15.79:42034) 2023-06-23T18:37:15.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.513+0000 7f2b29d9b700 1 -- 172.21.15.79:0/96852412 learned_addr learned my addr 172.21.15.79:0/96852412 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:15.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.513+0000 7f2b2ad9d700 1 --2- 172.21.15.79:0/96852412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2c0aee30 0x7f2b2c15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.513+0000 7f2b29d9b700 1 -- 172.21.15.79:0/96852412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2c0aee30 msgr2=0x7f2b2c15cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:15.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.513+0000 7f2b29d9b700 1 --2- 172.21.15.79:0/96852412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2c0aee30 0x7f2b2c15cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.513+0000 7f2b29d9b700 1 -- 172.21.15.79:0/96852412 >> v1:172.21.15.155:6789/0 conn(0x7f2b2c071490 legacy=0x7f2b2c071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.513+0000 7f2b29d9b700 1 -- 172.21.15.79:0/96852412 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c160b40 con 0x7f2b2c0b1270 2023-06-23T18:37:15.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.513+0000 7f2b29d9b700 1 --2- 172.21.15.79:0/96852412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b2c0b1270 0x7f2b2c15c9f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c0098f0 tx=0x7f2b1c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2df43b534329a1c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:15.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.513+0000 7f2b2959a700 1 -- 172.21.15.79:0/96852412 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00b8a0 con 0x7f2b2c0b1270 2023-06-23T18:37:15.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.513+0000 7f2b2959a700 1 -- 172.21.15.79:0/96852412 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c00ba00 con 0x7f2b2c0b1270 2023-06-23T18:37:15.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.513+0000 7f2b2959a700 1 -- 172.21.15.79:0/96852412 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00bd90 con 0x7f2b2c0b1270 2023-06-23T18:37:15.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.514+0000 7f2b30ca9700 1 -- 172.21.15.79:0/96852412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b2c0b1270 msgr2=0x7f2b2c15c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.514+0000 7f2b30ca9700 1 --2- 172.21.15.79:0/96852412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b2c0b1270 0x7f2b2c15c9f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c0098f0 tx=0x7f2b1c009c00 comp rx=0 tx=0).stop 2023-06-23T18:37:15.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.514+0000 7f2b30ca9700 1 -- 172.21.15.79:0/96852412 shutdown_connections 2023-06-23T18:37:15.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.514+0000 7f2b30ca9700 1 --2- 172.21.15.79:0/96852412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2c0aee30 0x7f2b2c15cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.514+0000 7f2b30ca9700 1 --2- 172.21.15.79:0/96852412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b2c0b1270 0x7f2b2c15c9f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.514+0000 7f2b30ca9700 1 -- 172.21.15.79:0/96852412 >> 172.21.15.79:0/96852412 conn(0x7f2b2c06d660 msgr2=0x7f2b2c06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:15.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.514+0000 7f2b30ca9700 1 -- 172.21.15.79:0/96852412 shutdown_connections 2023-06-23T18:37:15.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.514+0000 7f2b30ca9700 1 -- 172.21.15.79:0/96852412 wait complete. 2023-06-23T18:37:15.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.515+0000 7f2b30ca9700 1 Processor -- start 2023-06-23T18:37:15.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.515+0000 7f2b30ca9700 1 -- start start 2023-06-23T18:37:15.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b30ca9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b2c071490 0x7f2b2c0b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b30ca9700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2c0aee30 0x7f2b2c0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b30ca9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b2c0b1270 0x7f2b2c0c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b30ca9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0b3960 con 0x7f2b2c071490 2023-06-23T18:37:15.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b30ca9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0c8b80 con 0x7f2b2c0b1270 2023-06-23T18:37:15.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b30ca9700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0c8e60 con 0x7f2b2c0aee30 2023-06-23T18:37:15.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b2a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b2c071490 0x7f2b2c0b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b2a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b2c071490 0x7f2b2c0b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42040/0 (socket says 172.21.15.79:42040) 2023-06-23T18:37:15.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b2a59c700 1 -- 172.21.15.79:0/2521528495 learned_addr learned my addr 172.21.15.79:0/2521528495 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:15.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b29d9b700 1 --2- 172.21.15.79:0/2521528495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2c0aee30 0x7f2b2c0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b2ad9d700 1 --2- 172.21.15.79:0/2521528495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b2c0b1270 0x7f2b2c0c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b2a59c700 1 -- 172.21.15.79:0/2521528495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2c0aee30 msgr2=0x7f2b2c0b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b2a59c700 1 --2- 172.21.15.79:0/2521528495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2c0aee30 0x7f2b2c0b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b2a59c700 1 -- 172.21.15.79:0/2521528495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b2c0b1270 msgr2=0x7f2b2c0c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b2a59c700 1 --2- 172.21.15.79:0/2521528495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b2c0b1270 0x7f2b2c0c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b2a59c700 1 -- 172.21.15.79:0/2521528495 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c009550 con 0x7f2b2c071490 2023-06-23T18:37:15.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.516+0000 7f2b2a59c700 1 --2- 172.21.15.79:0/2521528495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b2c071490 0x7f2b2c0b3f90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000c450 tx=0x7f2b2000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:15.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.517+0000 7f2b1affd700 1 -- 172.21.15.79:0/2521528495 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b200052d0 con 0x7f2b2c071490 2023-06-23T18:37:15.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.517+0000 7f2b1affd700 1 -- 172.21.15.79:0/2521528495 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b20005430 con 0x7f2b2c071490 2023-06-23T18:37:15.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.517+0000 7f2b30ca9700 1 -- 172.21.15.79:0/2521528495 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2c0c9140 con 0x7f2b2c071490 2023-06-23T18:37:15.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.517+0000 7f2b1affd700 1 -- 172.21.15.79:0/2521528495 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b200038c0 con 0x7f2b2c071490 2023-06-23T18:37:15.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.517+0000 7f2b30ca9700 1 -- 172.21.15.79:0/2521528495 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c0c9700 con 0x7f2b2c071490 2023-06-23T18:37:15.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.518+0000 7f2b30ca9700 1 -- 172.21.15.79:0/2521528495 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2c0b4b30 con 0x7f2b2c071490 2023-06-23T18:37:15.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.518+0000 7f2b1affd700 1 -- 172.21.15.79:0/2521528495 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2b20003a20 con 0x7f2b2c071490 2023-06-23T18:37:15.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.518+0000 7f2b1affd700 1 --2- 172.21.15.79:0/2521528495 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2b0c06ad10 0x7f2b0c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.518+0000 7f2b1affd700 1 -- 172.21.15.79:0/2521528495 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b20088820 con 0x7f2b2c071490 2023-06-23T18:37:15.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.518+0000 7f2b29d9b700 1 --2- 172.21.15.79:0/2521528495 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2b0c06ad10 0x7f2b0c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.519+0000 7f2b29d9b700 1 --2- 172.21.15.79:0/2521528495 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2b0c06ad10 0x7f2b0c06d1c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c0098f0 tx=0x7f2b1c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:15.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.523+0000 7f2b1affd700 1 -- 172.21.15.79:0/2521528495 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2b200538d0 con 0x7f2b2c071490 2023-06-23T18:37:15.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.712+0000 7f2b30ca9700 1 -- 172.21.15.79:0/2521528495 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b2c0cdfc0 con 0x7f2b2c071490 2023-06-23T18:37:15.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.713+0000 7f2b1affd700 1 -- 172.21.15.79:0/2521528495 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+18305 (secure 0 0 0) 0x7f2b20056ef0 con 0x7f2b2c071490 2023-06-23T18:37:15.718 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:15.719 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":34,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:36:59.547297+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:36:56.964534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-23T18:36:57.507393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:37:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.715+0000 7f2b30ca9700 1 -- 172.21.15.79:0/2521528495 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2b0c06ad10 msgr2=0x7f2b0c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.715+0000 7f2b30ca9700 1 --2- 172.21.15.79:0/2521528495 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2b0c06ad10 0x7f2b0c06d1c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c0098f0 tx=0x7f2b1c008030 comp rx=0 tx=0).stop 2023-06-23T18:37:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.715+0000 7f2b30ca9700 1 -- 172.21.15.79:0/2521528495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b2c071490 msgr2=0x7f2b2c0b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.715+0000 7f2b30ca9700 1 --2- 172.21.15.79:0/2521528495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b2c071490 0x7f2b2c0b3f90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000c450 tx=0x7f2b2000c810 comp rx=0 tx=0).stop 2023-06-23T18:37:15.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.715+0000 7f2b30ca9700 1 -- 172.21.15.79:0/2521528495 shutdown_connections 2023-06-23T18:37:15.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.715+0000 7f2b30ca9700 1 --2- 172.21.15.79:0/2521528495 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2b0c06ad10 0x7f2b0c06d1c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.715+0000 7f2b30ca9700 1 --2- 172.21.15.79:0/2521528495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b2c0b1270 0x7f2b2c0c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.715+0000 7f2b30ca9700 1 --2- 172.21.15.79:0/2521528495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b2c071490 0x7f2b2c0b3f90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.715+0000 7f2b30ca9700 1 --2- 172.21.15.79:0/2521528495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2c0aee30 0x7f2b2c0b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.715+0000 7f2b30ca9700 1 -- 172.21.15.79:0/2521528495 >> 172.21.15.79:0/2521528495 conn(0x7f2b2c06d660 msgr2=0x7f2b2c157150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:15.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.715+0000 7f2b30ca9700 1 -- 172.21.15.79:0/2521528495 shutdown_connections 2023-06-23T18:37:15.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.715+0000 7f2b30ca9700 1 -- 172.21.15.79:0/2521528495 wait complete. 2023-06-23T18:37:15.732 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-06-23T18:37:15.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:37:15.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.848+0000 7f3981f3b700 1 Processor -- start 2023-06-23T18:37:15.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.848+0000 7f3981f3b700 1 -- start start 2023-06-23T18:37:15.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.848+0000 7f3981f3b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f397c06f0a0 0x7f397c164000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.848+0000 7f3981f3b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397c06df40 con 0x7f397c0ccd40 2023-06-23T18:37:15.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.848+0000 7f3981f3b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397c0cde00 con 0x7f397c159150 2023-06-23T18:37:15.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.848+0000 7f3981f3b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f397c0cdc00 con 0x7f397c06f0a0 2023-06-23T18:37:15.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.849+0000 7f397bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f397c06f0a0 0x7f397c164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.849+0000 7f397affd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f397c0ccd40 0x7f397c06e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45090/0 (socket says 172.21.15.79:45090) 2023-06-23T18:37:15.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.849+0000 7f397affd700 1 -- 172.21.15.79:0/602188577 learned_addr learned my addr 172.21.15.79:0/602188577 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:15.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.849+0000 7f397bfff700 1 -- 172.21.15.79:0/602188577 >> v1:172.21.15.155:6789/0 conn(0x7f397c159150 legacy=0x7f397c0c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:15.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.849+0000 7f397bfff700 1 -- 172.21.15.79:0/602188577 >> v1:172.21.15.79:6789/0 conn(0x7f397c0ccd40 legacy=0x7f397c06e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.849+0000 7f397bfff700 1 -- 172.21.15.79:0/602188577 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c0ce0e0 con 0x7f397c06f0a0 2023-06-23T18:37:15.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.849+0000 7f397bfff700 1 --2- 172.21.15.79:0/602188577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397c06f0a0 0x7f397c164000 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3974009d90 tx=0x7f397400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c159238a80f77c5a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:15.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.849+0000 7f397a7fc700 1 -- 172.21.15.79:0/602188577 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397400e070 con 0x7f397c06f0a0 2023-06-23T18:37:15.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.849+0000 7f397a7fc700 1 -- 172.21.15.79:0/602188577 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397400bb90 con 0x7f397c06f0a0 2023-06-23T18:37:15.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.849+0000 7f397a7fc700 1 -- 172.21.15.79:0/602188577 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974003a20 con 0x7f397c06f0a0 2023-06-23T18:37:15.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.850+0000 7f3981f3b700 1 -- 172.21.15.79:0/602188577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397c06f0a0 msgr2=0x7f397c164000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.850+0000 7f3981f3b700 1 --2- 172.21.15.79:0/602188577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397c06f0a0 0x7f397c164000 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3974009d90 tx=0x7f397400b330 comp rx=0 tx=0).stop 2023-06-23T18:37:15.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.850+0000 7f3981f3b700 1 -- 172.21.15.79:0/602188577 shutdown_connections 2023-06-23T18:37:15.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.850+0000 7f3981f3b700 1 --2- 172.21.15.79:0/602188577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397c06f0a0 0x7f397c164000 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.850+0000 7f3981f3b700 1 -- 172.21.15.79:0/602188577 >> 172.21.15.79:0/602188577 conn(0x7f397c0b17e0 msgr2=0x7f397c06cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:15.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.850+0000 7f3981f3b700 1 -- 172.21.15.79:0/602188577 shutdown_connections 2023-06-23T18:37:15.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.850+0000 7f3981f3b700 1 -- 172.21.15.79:0/602188577 wait complete. 2023-06-23T18:37:15.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.851+0000 7f3981f3b700 1 Processor -- start 2023-06-23T18:37:15.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.851+0000 7f3981f3b700 1 -- start start 2023-06-23T18:37:15.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f3981f3b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f397c06f0a0 0x7f397c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f3981f3b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397c0ccd40 0x7f397c15c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f3981f3b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f397c159150 0x7f397c070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f3981f3b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c0c8950 con 0x7f397c0ccd40 2023-06-23T18:37:15.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f3981f3b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c0c8bf0 con 0x7f397c159150 2023-06-23T18:37:15.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f3981f3b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397c0c8ed0 con 0x7f397c06f0a0 2023-06-23T18:37:15.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f397affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397c0ccd40 0x7f397c15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f397affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397c0ccd40 0x7f397c15c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42056/0 (socket says 172.21.15.79:42056) 2023-06-23T18:37:15.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f397affd700 1 -- 172.21.15.79:0/2601557977 learned_addr learned my addr 172.21.15.79:0/2601557977 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:15.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f397bfff700 1 --2- 172.21.15.79:0/2601557977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f397c159150 0x7f397c070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f397affd700 1 -- 172.21.15.79:0/2601557977 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f397c06f0a0 msgr2=0x7f397c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f397affd700 1 --2- 172.21.15.79:0/2601557977 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f397c06f0a0 0x7f397c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f397affd700 1 -- 172.21.15.79:0/2601557977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f397c159150 msgr2=0x7f397c070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:15.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f397affd700 1 --2- 172.21.15.79:0/2601557977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f397c159150 0x7f397c070bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:15.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f397affd700 1 -- 172.21.15.79:0/2601557977 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3974009a70 con 0x7f397c0ccd40 2023-06-23T18:37:15.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f397affd700 1 --2- 172.21.15.79:0/2601557977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397c0ccd40 0x7f397c15c370 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f396c00c500 tx=0x7f396c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:15.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.852+0000 7f3963fff700 1 -- 172.21.15.79:0/2601557977 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c005d70 con 0x7f397c0ccd40 2023-06-23T18:37:15.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.853+0000 7f3963fff700 1 -- 172.21.15.79:0/2601557977 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396c0070d0 con 0x7f397c0ccd40 2023-06-23T18:37:15.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.853+0000 7f3981f3b700 1 -- 172.21.15.79:0/2601557977 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397c0c91b0 con 0x7f397c0ccd40 2023-06-23T18:37:15.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.853+0000 7f3963fff700 1 -- 172.21.15.79:0/2601557977 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c007460 con 0x7f397c0ccd40 2023-06-23T18:37:15.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.853+0000 7f3981f3b700 1 -- 172.21.15.79:0/2601557977 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c0c62e0 con 0x7f397c0ccd40 2023-06-23T18:37:15.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.853+0000 7f3981f3b700 1 -- 172.21.15.79:0/2601557977 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397c0c9520 con 0x7f397c0ccd40 2023-06-23T18:37:15.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.854+0000 7f3963fff700 1 -- 172.21.15.79:0/2601557977 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f396c0075c0 con 0x7f397c0ccd40 2023-06-23T18:37:15.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.854+0000 7f3963fff700 1 --2- 172.21.15.79:0/2601557977 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f396806ace0 0x7f396806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:15.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.854+0000 7f3963fff700 1 -- 172.21.15.79:0/2601557977 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f396c088990 con 0x7f397c0ccd40 2023-06-23T18:37:15.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.854+0000 7f397b7fe700 1 --2- 172.21.15.79:0/2601557977 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f396806ace0 0x7f396806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:15.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.855+0000 7f397b7fe700 1 --2- 172.21.15.79:0/2601557977 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f396806ace0 0x7f396806d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f397c153060 tx=0x7f3970007a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:15.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:15.859+0000 7f3963fff700 1 -- 172.21.15.79:0/2601557977 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f396c053a50 con 0x7f397c0ccd40 2023-06-23T18:37:16.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.088+0000 7f3981f3b700 1 -- 172.21.15.79:0/2601557977 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f397c1ef790 con 0x7f397c0ccd40 2023-06-23T18:37:16.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.089+0000 7f3963fff700 1 -- 172.21.15.79:0/2601557977 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4677 (secure 0 0 0) 0x7f396c057070 con 0x7f397c0ccd40 2023-06-23T18:37:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:16.094 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3101032943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":3101032943},{"type":"v1","addr":"172.21.15.79:6837","nonce":3101032943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2056827644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2056827644},{"type":"v1","addr":"172.21.15.155:6833","nonce":2056827644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1835571378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1835571378},{"type":"v1","addr":"172.21.15.155:6835","nonce":1835571378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:36:58.545517+0000","modified":"2023-06-23T18:37:01.557968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6836/1030786659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1030786659},{"type":"v1","addr":"172.21.15.79:6836","nonce":1030786659}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-23T18:37:16.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.090+0000 7f3981f3b700 1 -- 172.21.15.79:0/2601557977 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f396806ace0 msgr2=0x7f396806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.090+0000 7f3981f3b700 1 --2- 172.21.15.79:0/2601557977 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f396806ace0 0x7f396806d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f397c153060 tx=0x7f3970007a40 comp rx=0 tx=0).stop 2023-06-23T18:37:16.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.090+0000 7f3981f3b700 1 -- 172.21.15.79:0/2601557977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397c0ccd40 msgr2=0x7f397c15c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.090+0000 7f3981f3b700 1 --2- 172.21.15.79:0/2601557977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397c0ccd40 0x7f397c15c370 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f396c00c500 tx=0x7f396c00c8c0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.091+0000 7f3981f3b700 1 -- 172.21.15.79:0/2601557977 shutdown_connections 2023-06-23T18:37:16.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.091+0000 7f3981f3b700 1 --2- 172.21.15.79:0/2601557977 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f396806ace0 0x7f396806d190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.091+0000 7f3981f3b700 1 --2- 172.21.15.79:0/2601557977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f397c159150 0x7f397c070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.091+0000 7f3981f3b700 1 --2- 172.21.15.79:0/2601557977 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f397c06f0a0 0x7f397c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.091+0000 7f3981f3b700 1 --2- 172.21.15.79:0/2601557977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397c0ccd40 0x7f397c15c370 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.091+0000 7f3981f3b700 1 -- 172.21.15.79:0/2601557977 >> 172.21.15.79:0/2601557977 conn(0x7f397c0b17e0 msgr2=0x7f397c0af640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:16.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.091+0000 7f3981f3b700 1 -- 172.21.15.79:0/2601557977 shutdown_connections 2023-06-23T18:37:16.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.091+0000 7f3981f3b700 1 -- 172.21.15.79:0/2601557977 wait complete. 2023-06-23T18:37:16.097 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 16 2023-06-23T18:37:16.107 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:37:16.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.221+0000 7f5a6c90b700 1 Processor -- start 2023-06-23T18:37:16.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.221+0000 7f5a6c90b700 1 -- start start 2023-06-23T18:37:16.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.222+0000 7f5a6c90b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a680b83c0 0x7f5a680cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.222+0000 7f5a6c90b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5a680d3710 0x7f5a680cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.222+0000 7f5a6c90b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a680cfee0 con 0x7f5a680cfa90 2023-06-23T18:37:16.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.222+0000 7f5a6c90b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a680d0a80 con 0x7f5a680d3710 2023-06-23T18:37:16.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.222+0000 7f5a6c90b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680b8790 con 0x7f5a680b83c0 2023-06-23T18:37:16.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.222+0000 7f5a65d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5a680d3710 0x7f5a680cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.222+0000 7f5a66d9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5a680cfa90 0x7f5a6815feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45104/0 (socket says 172.21.15.79:45104) 2023-06-23T18:37:16.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.222+0000 7f5a66d9d700 1 -- 172.21.15.79:0/900262410 learned_addr learned my addr 172.21.15.79:0/900262410 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:16.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.223+0000 7f5a6659c700 1 --2- 172.21.15.79:0/900262410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a680b83c0 0x7f5a680cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.223+0000 7f5a65d9b700 1 -- 172.21.15.79:0/900262410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a680b83c0 msgr2=0x7f5a680cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:16.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.223+0000 7f5a65d9b700 1 --2- 172.21.15.79:0/900262410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a680b83c0 0x7f5a680cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.223+0000 7f5a65d9b700 1 -- 172.21.15.79:0/900262410 >> v1:172.21.15.79:6789/0 conn(0x7f5a680cfa90 legacy=0x7f5a6815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.223+0000 7f5a65d9b700 1 -- 172.21.15.79:0/900262410 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a680b4e70 con 0x7f5a680d3710 2023-06-23T18:37:16.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.223+0000 7f5a6559a700 1 -- 172.21.15.79:0/900262410 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514645966 0 0) 0x7f5a60001490 con 0x7f5a680cfa90 2023-06-23T18:37:16.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.223+0000 7f5a65d9b700 1 --2- 172.21.15.79:0/900262410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a680d3710 0x7f5a680cf550 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009840 tx=0x7f5a5c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a2abb168e53485 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:16.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.223+0000 7f5a6559a700 1 -- 172.21.15.79:0/900262410 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c011830 con 0x7f5a680d3710 2023-06-23T18:37:16.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.223+0000 7f5a6559a700 1 -- 172.21.15.79:0/900262410 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c011990 con 0x7f5a680d3710 2023-06-23T18:37:16.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.223+0000 7f5a6559a700 1 -- 172.21.15.79:0/900262410 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c011d20 con 0x7f5a680d3710 2023-06-23T18:37:16.232 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.224+0000 7f5a6c90b700 1 -- 172.21.15.79:0/900262410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a680d3710 msgr2=0x7f5a680cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.232 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.224+0000 7f5a6c90b700 1 --2- 172.21.15.79:0/900262410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a680d3710 0x7f5a680cf550 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009840 tx=0x7f5a5c009b50 comp rx=0 tx=0).stop 2023-06-23T18:37:16.232 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.224+0000 7f5a6c90b700 1 -- 172.21.15.79:0/900262410 shutdown_connections 2023-06-23T18:37:16.232 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.224+0000 7f5a6c90b700 1 --2- 172.21.15.79:0/900262410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a680b83c0 0x7f5a680cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.224+0000 7f5a6c90b700 1 --2- 172.21.15.79:0/900262410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a680d3710 0x7f5a680cf550 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.224+0000 7f5a6c90b700 1 -- 172.21.15.79:0/900262410 >> 172.21.15.79:0/900262410 conn(0x7f5a6806b350 msgr2=0x7f5a6806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:16.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.224+0000 7f5a6c90b700 1 -- 172.21.15.79:0/900262410 shutdown_connections 2023-06-23T18:37:16.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.224+0000 7f5a6c90b700 1 -- 172.21.15.79:0/900262410 wait complete. 2023-06-23T18:37:16.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.225+0000 7f5a6c90b700 1 Processor -- start 2023-06-23T18:37:16.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.225+0000 7f5a6c90b700 1 -- start start 2023-06-23T18:37:16.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6c90b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a680b83c0 0x7f5a68153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6c90b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a680cfa90 0x7f5a680d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6c90b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a680d3710 0x7f5a680d1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6c90b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a68153840 con 0x7f5a680b83c0 2023-06-23T18:37:16.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6c90b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a681e8c30 con 0x7f5a680cfa90 2023-06-23T18:37:16.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6c90b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680cb410 con 0x7f5a680d3710 2023-06-23T18:37:16.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a680b83c0 0x7f5a68153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a680b83c0 0x7f5a68153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42066/0 (socket says 172.21.15.79:42066) 2023-06-23T18:37:16.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6659c700 1 -- 172.21.15.79:0/1124035708 learned_addr learned my addr 172.21.15.79:0/1124035708 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:16.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6659c700 1 -- 172.21.15.79:0/1124035708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a680d3710 msgr2=0x7f5a680d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6659c700 1 --2- 172.21.15.79:0/1124035708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a680d3710 0x7f5a680d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6659c700 1 -- 172.21.15.79:0/1124035708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a680cfa90 msgr2=0x7f5a680d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6659c700 1 --2- 172.21.15.79:0/1124035708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a680cfa90 0x7f5a680d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6659c700 1 -- 172.21.15.79:0/1124035708 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c0094a0 con 0x7f5a680b83c0 2023-06-23T18:37:16.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6659c700 1 --2- 172.21.15.79:0/1124035708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a680b83c0 0x7f5a68153300 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800d830 tx=0x7f5a5800dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:16.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a56ffd700 1 -- 172.21.15.79:0/1124035708 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a580125f0 con 0x7f5a680b83c0 2023-06-23T18:37:16.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a56ffd700 1 -- 172.21.15.79:0/1124035708 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a58012750 con 0x7f5a680b83c0 2023-06-23T18:37:16.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6c90b700 1 -- 172.21.15.79:0/1124035708 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a681e8950 con 0x7f5a680b83c0 2023-06-23T18:37:16.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a56ffd700 1 -- 172.21.15.79:0/1124035708 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58012ae0 con 0x7f5a680b83c0 2023-06-23T18:37:16.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.226+0000 7f5a6c90b700 1 -- 172.21.15.79:0/1124035708 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a680cb0a0 con 0x7f5a680b83c0 2023-06-23T18:37:16.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.227+0000 7f5a6c90b700 1 -- 172.21.15.79:0/1124035708 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a680cb230 con 0x7f5a680b83c0 2023-06-23T18:37:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.228+0000 7f5a56ffd700 1 -- 172.21.15.79:0/1124035708 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a58012c40 con 0x7f5a680b83c0 2023-06-23T18:37:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.228+0000 7f5a56ffd700 1 --2- 172.21.15.79:0/1124035708 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.228+0000 7f5a56ffd700 1 -- 172.21.15.79:0/1124035708 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5a58089800 con 0x7f5a680b83c0 2023-06-23T18:37:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.228+0000 7f5a65d9b700 1 --2- 172.21.15.79:0/1124035708 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.229+0000 7f5a65d9b700 1 --2- 172.21.15.79:0/1124035708 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5a4806ace0 0x7f5a4806d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5a680d3b20 tx=0x7f5a5c010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:16.238 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.232+0000 7f5a56ffd700 1 -- 172.21.15.79:0/1124035708 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5a580548b0 con 0x7f5a680b83c0 2023-06-23T18:37:16.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.462+0000 7f5a6c90b700 1 -- 172.21.15.79:0/1124035708 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a6815ad90 con 0x7f5a680b83c0 2023-06-23T18:37:16.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.462+0000 7f5a56ffd700 1 -- 172.21.15.79:0/1124035708 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4677 (secure 0 0 0) 0x7f5a58057ed0 con 0x7f5a680b83c0 2023-06-23T18:37:16.468 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:16.468 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3101032943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":3101032943},{"type":"v1","addr":"172.21.15.79:6837","nonce":3101032943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2056827644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2056827644},{"type":"v1","addr":"172.21.15.155:6833","nonce":2056827644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1835571378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1835571378},{"type":"v1","addr":"172.21.15.155:6835","nonce":1835571378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:36:58.545517+0000","modified":"2023-06-23T18:37:01.557968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6836/1030786659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1030786659},{"type":"v1","addr":"172.21.15.79:6836","nonce":1030786659}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-23T18:37:16.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.464+0000 7f5a6c90b700 1 -- 172.21.15.79:0/1124035708 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5a4806ace0 msgr2=0x7f5a4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.464+0000 7f5a6c90b700 1 --2- 172.21.15.79:0/1124035708 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5a4806ace0 0x7f5a4806d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5a680d3b20 tx=0x7f5a5c010000 comp rx=0 tx=0).stop 2023-06-23T18:37:16.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.464+0000 7f5a6c90b700 1 -- 172.21.15.79:0/1124035708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a680b83c0 msgr2=0x7f5a68153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.464+0000 7f5a6c90b700 1 --2- 172.21.15.79:0/1124035708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a680b83c0 0x7f5a68153300 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800d830 tx=0x7f5a5800dbf0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.464+0000 7f5a6c90b700 1 -- 172.21.15.79:0/1124035708 shutdown_connections 2023-06-23T18:37:16.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.464+0000 7f5a6c90b700 1 --2- 172.21.15.79:0/1124035708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a680d3710 0x7f5a680d1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.464+0000 7f5a6c90b700 1 --2- 172.21.15.79:0/1124035708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a680b83c0 0x7f5a68153300 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.464+0000 7f5a6c90b700 1 --2- 172.21.15.79:0/1124035708 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.464+0000 7f5a6c90b700 1 --2- 172.21.15.79:0/1124035708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a680cfa90 0x7f5a680d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.464+0000 7f5a6c90b700 1 -- 172.21.15.79:0/1124035708 >> 172.21.15.79:0/1124035708 conn(0x7f5a6806b350 msgr2=0x7f5a6815bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:16.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.464+0000 7f5a6c90b700 1 -- 172.21.15.79:0/1124035708 shutdown_connections 2023-06-23T18:37:16.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.464+0000 7f5a6c90b700 1 -- 172.21.15.79:0/1124035708 wait complete. 2023-06-23T18:37:16.471 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 16 2023-06-23T18:37:16.481 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:37:16.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.595+0000 7f2f41dd6700 1 Processor -- start 2023-06-23T18:37:16.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.596+0000 7f2f41dd6700 1 -- start start 2023-06-23T18:37:16.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.596+0000 7f2f41dd6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f3c0cdad0 0x7f2f3c0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.596+0000 7f2f41dd6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2f3c0ce620 0x7f2f3c0cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.596+0000 7f2f41dd6700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c15cd50 con 0x7f2f3c159170 2023-06-23T18:37:16.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.596+0000 7f2f41dd6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f3c0cea80 con 0x7f2f3c0ce620 2023-06-23T18:37:16.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.596+0000 7f2f41dd6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c15cfa0 con 0x7f2f3c0cdad0 2023-06-23T18:37:16.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.596+0000 7f2f3bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2f3c0ce620 0x7f2f3c0cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.596+0000 7f2f3bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2f3c0ce620 0x7f2f3c0cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42078/0 (socket says 172.21.15.79:42078) 2023-06-23T18:37:16.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.597+0000 7f2f3bfff700 1 -- 172.21.15.79:0/2062191488 learned_addr learned my addr 172.21.15.79:0/2062191488 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:16.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.597+0000 7f2f3affd700 1 --2- 172.21.15.79:0/2062191488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f3c0cdad0 0x7f2f3c0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.597+0000 7f2f3bfff700 1 -- 172.21.15.79:0/2062191488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f3c0cdad0 msgr2=0x7f2f3c0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:16.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.597+0000 7f2f3bfff700 1 --2- 172.21.15.79:0/2062191488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f3c0cdad0 0x7f2f3c0ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.597+0000 7f2f3bfff700 1 -- 172.21.15.79:0/2062191488 >> v1:172.21.15.155:6789/0 conn(0x7f2f3c159170 legacy=0x7f2f3c0cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:16.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.597+0000 7f2f3bfff700 1 -- 172.21.15.79:0/2062191488 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c15d280 con 0x7f2f3c0ce620 2023-06-23T18:37:16.606 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.597+0000 7f2f3affd700 1 --2- 172.21.15.79:0/2062191488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f3c0cdad0 0x7f2f3c0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:16.606 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.597+0000 7f2f3bfff700 1 --2- 172.21.15.79:0/2062191488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c0ce620 0x7f2f3c0cc1d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009e50 tx=0x7f2f3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80cd5194cc1ae216 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:16.606 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.597+0000 7f2f3a7fc700 1 -- 172.21.15.79:0/2062191488 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3400e070 con 0x7f2f3c0ce620 2023-06-23T18:37:16.606 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.597+0000 7f2f3a7fc700 1 -- 172.21.15.79:0/2062191488 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3400b9c0 con 0x7f2f3c0ce620 2023-06-23T18:37:16.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.597+0000 7f2f3a7fc700 1 -- 172.21.15.79:0/2062191488 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3400bd70 con 0x7f2f3c0ce620 2023-06-23T18:37:16.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.598+0000 7f2f41dd6700 1 -- 172.21.15.79:0/2062191488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c0ce620 msgr2=0x7f2f3c0cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.598+0000 7f2f41dd6700 1 --2- 172.21.15.79:0/2062191488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c0ce620 0x7f2f3c0cc1d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009e50 tx=0x7f2f3400b330 comp rx=0 tx=0).stop 2023-06-23T18:37:16.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.598+0000 7f2f41dd6700 1 -- 172.21.15.79:0/2062191488 shutdown_connections 2023-06-23T18:37:16.607 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.598+0000 7f2f41dd6700 1 --2- 172.21.15.79:0/2062191488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c0ce620 0x7f2f3c0cc1d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.598+0000 7f2f41dd6700 1 --2- 172.21.15.79:0/2062191488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f3c0cdad0 0x7f2f3c0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.598+0000 7f2f41dd6700 1 -- 172.21.15.79:0/2062191488 >> 172.21.15.79:0/2062191488 conn(0x7f2f3c06d660 msgr2=0x7f2f3c06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:16.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.598+0000 7f2f41dd6700 1 -- 172.21.15.79:0/2062191488 shutdown_connections 2023-06-23T18:37:16.608 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.598+0000 7f2f41dd6700 1 -- 172.21.15.79:0/2062191488 wait complete. 2023-06-23T18:37:16.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.599+0000 7f2f41dd6700 1 Processor -- start 2023-06-23T18:37:16.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.599+0000 7f2f41dd6700 1 -- start start 2023-06-23T18:37:16.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f41dd6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f3c0cdad0 0x7f2f3c154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f41dd6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c0ce620 0x7f2f3c15c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f41dd6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f3c159170 0x7f2f3c0b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f41dd6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c0c4f70 con 0x7f2f3c0ce620 2023-06-23T18:37:16.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f41dd6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c154840 con 0x7f2f3c0cdad0 2023-06-23T18:37:16.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f41dd6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c0c51c0 con 0x7f2f3c159170 2023-06-23T18:37:16.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f3affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c0ce620 0x7f2f3c15c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f3affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c0ce620 0x7f2f3c15c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42082/0 (socket says 172.21.15.79:42082) 2023-06-23T18:37:16.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f3affd700 1 -- 172.21.15.79:0/969114344 learned_addr learned my addr 172.21.15.79:0/969114344 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:16.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f3b7fe700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f3c0cdad0 0x7f2f3c154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f3bfff700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f3c159170 0x7f2f3c0b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f3affd700 1 -- 172.21.15.79:0/969114344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f3c159170 msgr2=0x7f2f3c0b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f3affd700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f3c159170 0x7f2f3c0b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f3affd700 1 -- 172.21.15.79:0/969114344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f3c0cdad0 msgr2=0x7f2f3c154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f3affd700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f3c0cdad0 0x7f2f3c154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f3affd700 1 -- 172.21.15.79:0/969114344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f34009b30 con 0x7f2f3c0ce620 2023-06-23T18:37:16.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f3b7fe700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f3c0cdad0 0x7f2f3c154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:16.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f3bfff700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f3c159170 0x7f2f3c0b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:16.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f3affd700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c0ce620 0x7f2f3c15c050 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000c740 tx=0x7f2f3000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:16.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f23fff700 1 -- 172.21.15.79:0/969114344 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30004220 con 0x7f2f3c0ce620 2023-06-23T18:37:16.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f23fff700 1 -- 172.21.15.79:0/969114344 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f30004380 con 0x7f2f3c0ce620 2023-06-23T18:37:16.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f41dd6700 1 -- 172.21.15.79:0/969114344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c0c54a0 con 0x7f2f3c0ce620 2023-06-23T18:37:16.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f23fff700 1 -- 172.21.15.79:0/969114344 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f300036a0 con 0x7f2f3c0ce620 2023-06-23T18:37:16.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.600+0000 7f2f41dd6700 1 -- 172.21.15.79:0/969114344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c0c5a60 con 0x7f2f3c0ce620 2023-06-23T18:37:16.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.601+0000 7f2f41dd6700 1 -- 172.21.15.79:0/969114344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3c0cee90 con 0x7f2f3c0ce620 2023-06-23T18:37:16.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.602+0000 7f2f23fff700 1 -- 172.21.15.79:0/969114344 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2f30003800 con 0x7f2f3c0ce620 2023-06-23T18:37:16.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.602+0000 7f2f23fff700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2f2806ace0 0x7f2f2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.602+0000 7f2f23fff700 1 -- 172.21.15.79:0/969114344 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2f300897b0 con 0x7f2f3c0ce620 2023-06-23T18:37:16.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.602+0000 7f2f3b7fe700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2f2806ace0 0x7f2f2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.603+0000 7f2f3b7fe700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2f2806ace0 0x7f2f2806d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c000f60 tx=0x7f2f2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:16.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.607+0000 7f2f23fff700 1 -- 172.21.15.79:0/969114344 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2f30058300 con 0x7f2f3c0ce620 2023-06-23T18:37:16.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.794+0000 7f2f41dd6700 1 -- 172.21.15.79:0/969114344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f3c0cac10 con 0x7f2f3c0ce620 2023-06-23T18:37:16.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.795+0000 7f2f23fff700 1 -- 172.21.15.79:0/969114344 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+18305 (secure 0 0 0) 0x7f2f30057e90 con 0x7f2f3c0ce620 2023-06-23T18:37:16.801 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:16.801 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":34,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:36:59.547297+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:36:56.964534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-23T18:36:57.507393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:37:16.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.797+0000 7f2f41dd6700 1 -- 172.21.15.79:0/969114344 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2f2806ace0 msgr2=0x7f2f2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.797+0000 7f2f41dd6700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2f2806ace0 0x7f2f2806d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c000f60 tx=0x7f2f2c009040 comp rx=0 tx=0).stop 2023-06-23T18:37:16.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.797+0000 7f2f41dd6700 1 -- 172.21.15.79:0/969114344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c0ce620 msgr2=0x7f2f3c15c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.797+0000 7f2f41dd6700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c0ce620 0x7f2f3c15c050 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000c740 tx=0x7f2f3000cb00 comp rx=0 tx=0).stop 2023-06-23T18:37:16.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.797+0000 7f2f41dd6700 1 -- 172.21.15.79:0/969114344 shutdown_connections 2023-06-23T18:37:16.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.797+0000 7f2f41dd6700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f3c159170 0x7f2f3c0b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.797+0000 7f2f41dd6700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2f2806ace0 0x7f2f2806d190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.797+0000 7f2f41dd6700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f3c0cdad0 0x7f2f3c154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.797+0000 7f2f41dd6700 1 --2- 172.21.15.79:0/969114344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c0ce620 0x7f2f3c15c050 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.797+0000 7f2f41dd6700 1 -- 172.21.15.79:0/969114344 >> 172.21.15.79:0/969114344 conn(0x7f2f3c06d660 msgr2=0x7f2f3c0cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:16.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.797+0000 7f2f41dd6700 1 -- 172.21.15.79:0/969114344 shutdown_connections 2023-06-23T18:37:16.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.797+0000 7f2f41dd6700 1 -- 172.21.15.79:0/969114344 wait complete. 2023-06-23T18:37:16.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-06-23T18:37:16.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.932+0000 7fcc26735700 1 Processor -- start 2023-06-23T18:37:16.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.932+0000 7fcc26735700 1 -- start start 2023-06-23T18:37:16.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.933+0000 7fcc26735700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc20159170 0x7fcc200cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.933+0000 7fcc26735700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2015cd50 con 0x7fcc200ce7f0 2023-06-23T18:37:16.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.933+0000 7fcc26735700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2015cfa0 con 0x7fcc200cd900 2023-06-23T18:37:16.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.933+0000 7fcc26735700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc200cdd70 con 0x7fcc20159170 2023-06-23T18:37:16.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.933+0000 7fcc24cd2700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fcc200ce7f0 0x7fcc200cc1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45130/0 (socket says 172.21.15.79:45130) 2023-06-23T18:37:16.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.933+0000 7fcc24cd2700 1 -- 172.21.15.79:0/2678948461 learned_addr learned my addr 172.21.15.79:0/2678948461 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:16.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1ffff700 1 --2- 172.21.15.79:0/2678948461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc20159170 0x7fcc200cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087735929 0 0) 0x7fcc2015cd50 con 0x7fcc200ce7f0 2023-06-23T18:37:16.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc080035b0 con 0x7fcc200ce7f0 2023-06-23T18:37:16.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825115296 0 0) 0x7fcc2015cfa0 con 0x7fcc200cd900 2023-06-23T18:37:16.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc2015cd50 con 0x7fcc200cd900 2023-06-23T18:37:16.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3708497159 0 0) 0x7fcc080035b0 con 0x7fcc200ce7f0 2023-06-23T18:37:16.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc20159170 msgr2=0x7fcc200cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:16.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 --2- 172.21.15.79:0/2678948461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc20159170 0x7fcc200cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 >> v1:172.21.15.155:6789/0 conn(0x7fcc200cd900 legacy=0x7fcc200ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2015d280 con 0x7fcc200ce7f0 2023-06-23T18:37:16.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fcc180019f0 con 0x7fcc200ce7f0 2023-06-23T18:37:16.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 >> v1:172.21.15.79:6789/0 conn(0x7fcc200ce7f0 legacy=0x7fcc200cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 --2- 172.21.15.79:0/2678948461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc08003bc0 0x7fcc08003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 --2- 172.21.15.79:0/2678948461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc20159170 0x7fcc08004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 --2- 172.21.15.79:0/2678948461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc08004d90 0x7fcc08005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc200cdd70 con 0x7fcc08004d90 2023-06-23T18:37:16.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0800bb10 con 0x7fcc08003bc0 2023-06-23T18:37:16.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0800bcd0 con 0x7fcc20159170 2023-06-23T18:37:16.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc24cd2700 1 --2- 172.21.15.79:0/2678948461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc08004d90 0x7fcc08005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1ffff700 1 --2- 172.21.15.79:0/2678948461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc08003bc0 0x7fcc08003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc1f7fe700 1 --2- 172.21.15.79:0/2678948461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc20159170 0x7fcc08004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc24cd2700 1 -- 172.21.15.79:0/2678948461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc20159170 msgr2=0x7fcc08004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:16.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc24cd2700 1 --2- 172.21.15.79:0/2678948461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc20159170 0x7fcc08004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc24cd2700 1 -- 172.21.15.79:0/2678948461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc08003bc0 msgr2=0x7fcc08003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:16.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc24cd2700 1 --2- 172.21.15.79:0/2678948461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc08003bc0 0x7fcc08003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.934+0000 7fcc24cd2700 1 -- 172.21.15.79:0/2678948461 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc080035b0 con 0x7fcc08004d90 2023-06-23T18:37:16.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.935+0000 7fcc24cd2700 1 --2- 172.21.15.79:0/2678948461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc08004d90 0x7fcc08005200 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800a7a0 tx=0x7fcc08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe28e5386543dc5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:16.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.935+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc18012600 con 0x7fcc08004d90 2023-06-23T18:37:16.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.935+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc18012780 con 0x7fcc08004d90 2023-06-23T18:37:16.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.935+0000 7fcc1effd700 1 -- 172.21.15.79:0/2678948461 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc18012b30 con 0x7fcc08004d90 2023-06-23T18:37:16.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.935+0000 7fcc26735700 1 -- 172.21.15.79:0/2678948461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc08004d90 msgr2=0x7fcc08005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.935+0000 7fcc26735700 1 --2- 172.21.15.79:0/2678948461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc08004d90 0x7fcc08005200 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800a7a0 tx=0x7fcc08005880 comp rx=0 tx=0).stop 2023-06-23T18:37:16.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.935+0000 7fcc24cd2700 1 -- 172.21.15.79:0/2678948461 reap_dead start 2023-06-23T18:37:16.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.935+0000 7fcc26735700 1 -- 172.21.15.79:0/2678948461 shutdown_connections 2023-06-23T18:37:16.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.935+0000 7fcc26735700 1 -- 172.21.15.79:0/2678948461 >> 172.21.15.79:0/2678948461 conn(0x7fcc2006d660 msgr2=0x7fcc2006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:16.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.935+0000 7fcc26735700 1 -- 172.21.15.79:0/2678948461 shutdown_connections 2023-06-23T18:37:16.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.935+0000 7fcc26735700 1 -- 172.21.15.79:0/2678948461 wait complete. 2023-06-23T18:37:16.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.936+0000 7fcc26735700 1 Processor -- start 2023-06-23T18:37:16.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.937+0000 7fcc26735700 1 -- start start 2023-06-23T18:37:16.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.937+0000 7fcc26735700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc201505e0 0x7fcc201509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.937+0000 7fcc26735700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc2015b2c0 0x7fcc201e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.937+0000 7fcc26735700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2015b730 0x7fcc201e5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.937+0000 7fcc26735700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc20150ef0 con 0x7fcc2015b730 2023-06-23T18:37:16.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.937+0000 7fcc26735700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2015c280 con 0x7fcc2015b2c0 2023-06-23T18:37:16.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.937+0000 7fcc26735700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2015c560 con 0x7fcc201505e0 2023-06-23T18:37:16.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc24cd2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2015b730 0x7fcc201e5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc24cd2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2015b730 0x7fcc201e5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42102/0 (socket says 172.21.15.79:42102) 2023-06-23T18:37:16.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc24cd2700 1 -- 172.21.15.79:0/1087878205 learned_addr learned my addr 172.21.15.79:0/1087878205 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:16.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc1ffff700 1 --2- 172.21.15.79:0/1087878205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc201505e0 0x7fcc201509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc1f7fe700 1 --2- 172.21.15.79:0/1087878205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc2015b2c0 0x7fcc201e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc24cd2700 1 -- 172.21.15.79:0/1087878205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc201505e0 msgr2=0x7fcc201509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc24cd2700 1 --2- 172.21.15.79:0/1087878205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc201505e0 0x7fcc201509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc24cd2700 1 -- 172.21.15.79:0/1087878205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc2015b2c0 msgr2=0x7fcc201e4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:16.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc24cd2700 1 --2- 172.21.15.79:0/1087878205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc2015b2c0 0x7fcc201e4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:16.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc24cd2700 1 -- 172.21.15.79:0/1087878205 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc200b26c0 con 0x7fcc2015b730 2023-06-23T18:37:16.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc24cd2700 1 --2- 172.21.15.79:0/1087878205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2015b730 0x7fcc201e5130 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcc18015040 tx=0x7fcc18001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:16.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc1cff9700 1 -- 172.21.15.79:0/1087878205 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc18012610 con 0x7fcc2015b730 2023-06-23T18:37:16.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc1cff9700 1 -- 172.21.15.79:0/1087878205 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc180017b0 con 0x7fcc2015b730 2023-06-23T18:37:16.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc1cff9700 1 -- 172.21.15.79:0/1087878205 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc18027410 con 0x7fcc2015b730 2023-06-23T18:37:16.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc26735700 1 -- 172.21.15.79:0/1087878205 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc200b29a0 con 0x7fcc2015b730 2023-06-23T18:37:16.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.938+0000 7fcc26735700 1 -- 172.21.15.79:0/1087878205 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2015bea0 con 0x7fcc2015b730 2023-06-23T18:37:16.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.939+0000 7fcc26735700 1 -- 172.21.15.79:0/1087878205 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc200b2ec0 con 0x7fcc2015b730 2023-06-23T18:37:16.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.939+0000 7fcc1cff9700 1 -- 172.21.15.79:0/1087878205 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcc18019070 con 0x7fcc2015b730 2023-06-23T18:37:16.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.940+0000 7fcc1cff9700 1 --2- 172.21.15.79:0/1087878205 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcc0c06ad10 0x7fcc0c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:16.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.940+0000 7fcc1cff9700 1 -- 172.21.15.79:0/1087878205 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcc1808a710 con 0x7fcc2015b730 2023-06-23T18:37:16.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.940+0000 7fcc1ffff700 1 --2- 172.21.15.79:0/1087878205 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcc0c06ad10 0x7fcc0c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:16.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.940+0000 7fcc1ffff700 1 --2- 172.21.15.79:0/1087878205 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcc0c06ad10 0x7fcc0c06d1c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009c60 tx=0x7fcc14009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:16.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:16.944+0000 7fcc1cff9700 1 -- 172.21.15.79:0/1087878205 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcc200b2ec0 con 0x7fcc2015b730 2023-06-23T18:37:17.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:17.176+0000 7fcc26735700 1 -- 172.21.15.79:0/1087878205 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcc200b2ec0 con 0x7fcc2015b730 2023-06-23T18:37:17.971 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-06-23T18:37:17.966+0000 7f71a235a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-06-23T18:37:17.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:17.975+0000 7fcc1cff9700 1 -- 172.21.15.79:0/1087878205 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcc200b2ec0 con 0x7fcc2015b730 2023-06-23T18:37:17.981 INFO:tasks.ceph.mds.c.smithi079.stderr: -1442> 2023-06-23T18:37:02.128+0000 7f7fbcff8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-23T18:37:17.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:17.978+0000 7fcc26735700 1 -- 172.21.15.79:0/1087878205 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcc0c06ad10 msgr2=0x7fcc0c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:17.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:17.978+0000 7fcc26735700 1 --2- 172.21.15.79:0/1087878205 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcc0c06ad10 0x7fcc0c06d1c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009c60 tx=0x7fcc14009210 comp rx=0 tx=0).stop 2023-06-23T18:37:17.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:17.978+0000 7fcc26735700 1 -- 172.21.15.79:0/1087878205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2015b730 msgr2=0x7fcc201e5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:17.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:17.978+0000 7fcc26735700 1 --2- 172.21.15.79:0/1087878205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2015b730 0x7fcc201e5130 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcc18015040 tx=0x7fcc18001d00 comp rx=0 tx=0).stop 2023-06-23T18:37:17.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:17.978+0000 7fcc26735700 1 -- 172.21.15.79:0/1087878205 shutdown_connections 2023-06-23T18:37:17.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:17.978+0000 7fcc26735700 1 --2- 172.21.15.79:0/1087878205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2015b730 0x7fcc201e5130 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:17.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:17.978+0000 7fcc26735700 1 --2- 172.21.15.79:0/1087878205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc201505e0 0x7fcc201509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:17.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:17.978+0000 7fcc26735700 1 --2- 172.21.15.79:0/1087878205 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fcc0c06ad10 0x7fcc0c06d1c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:17.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:17.978+0000 7fcc26735700 1 --2- 172.21.15.79:0/1087878205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc2015b2c0 0x7fcc201e4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:17.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:17.978+0000 7fcc26735700 1 -- 172.21.15.79:0/1087878205 >> 172.21.15.79:0/1087878205 conn(0x7fcc2006d660 msgr2=0x7fcc20157850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:17.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:17.978+0000 7fcc26735700 1 -- 172.21.15.79:0/1087878205 shutdown_connections 2023-06-23T18:37:17.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:17.978+0000 7fcc26735700 1 -- 172.21.15.79:0/1087878205 wait complete. 2023-06-23T18:37:17.985 INFO:teuthology.orchestra.run.smithi079.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-06-23T18:37:18.002 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-06-23T18:37:18.019 INFO:tasks.ceph.mds.c.smithi079.stdout:starting mds.c at 2023-06-23T18:37:18.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.124+0000 7fd0e2a7b700 1 Processor -- start 2023-06-23T18:37:18.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.124+0000 7fd0e2a7b700 1 -- start start 2023-06-23T18:37:18.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.125+0000 7fd0e2a7b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc06b6b0 0x7fd0dc0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:18.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.125+0000 7fd0e2a7b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd0dc0cfc10 0x7fd0dc0cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:18.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.125+0000 7fd0e2a7b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0dc0b0890 con 0x7fd0dc15c840 2023-06-23T18:37:18.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.125+0000 7fd0e2a7b700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0dc0d0090 con 0x7fd0dc0cfc10 2023-06-23T18:37:18.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.125+0000 7fd0e2a7b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc0d0310 con 0x7fd0dc06b6b0 2023-06-23T18:37:18.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.125+0000 7fd0e0817700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd0dc15c840 0x7fd0dc06afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45166/0 (socket says 172.21.15.79:45166) 2023-06-23T18:37:18.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.125+0000 7fd0e0817700 1 -- 172.21.15.79:0/1745527773 learned_addr learned my addr 172.21.15.79:0/1745527773 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:18.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.125+0000 7fd0e1018700 1 --2- 172.21.15.79:0/1745527773 >> v2:172.21.15.155:3300/0 conn(0x7fd0dc0cfc10 0x7fd0dc0cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:18.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.125+0000 7fd0dbfff700 1 --2- 172.21.15.79:0/1745527773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc06b6b0 0x7fd0dc0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:18.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.125+0000 7fd0db7fe700 1 -- 172.21.15.79:0/1745527773 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171723230 0 0) 0x7fd0dc0b0890 con 0x7fd0dc15c840 2023-06-23T18:37:18.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.125+0000 7fd0db7fe700 1 -- 172.21.15.79:0/1745527773 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0cc0035b0 con 0x7fd0dc15c840 2023-06-23T18:37:18.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 -- 172.21.15.79:0/1745527773 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3848305340 0 0) 0x7fd0cc0035b0 con 0x7fd0dc15c840 2023-06-23T18:37:18.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 -- 172.21.15.79:0/1745527773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc06b6b0 msgr2=0x7fd0dc0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:18.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 --2- 172.21.15.79:0/1745527773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc06b6b0 0x7fd0dc0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:18.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 -- 172.21.15.79:0/1745527773 >> v2:172.21.15.155:3300/0 conn(0x7fd0dc0cfc10 msgr2=0x7fd0dc0cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:18.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 --2- 172.21.15.79:0/1745527773 >> v2:172.21.15.155:3300/0 conn(0x7fd0dc0cfc10 0x7fd0dc0cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:18.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 -- 172.21.15.79:0/1745527773 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc0b0ae0 con 0x7fd0dc15c840 2023-06-23T18:37:18.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 -- 172.21.15.79:0/1745527773 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fd0d00017f0 con 0x7fd0dc15c840 2023-06-23T18:37:18.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 -- 172.21.15.79:0/1745527773 >> v1:172.21.15.79:6789/0 conn(0x7fd0dc15c840 legacy=0x7fd0dc06afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:18.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 --2- 172.21.15.79:0/1745527773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0cc003a00 0x7fd0cc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:18.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 --2- 172.21.15.79:0/1745527773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc06b6b0 0x7fd0cc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:18.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 --2- 172.21.15.79:0/1745527773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0cc004d90 0x7fd0cc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:18.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 -- 172.21.15.79:0/1745527773 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc0d0090 con 0x7fd0cc003a00 2023-06-23T18:37:18.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 -- 172.21.15.79:0/1745527773 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc0d0310 con 0x7fd0cc004d90 2023-06-23T18:37:18.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 -- 172.21.15.79:0/1745527773 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc00ba10 con 0x7fd0dc06b6b0 2023-06-23T18:37:18.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0e1018700 1 --2- 172.21.15.79:0/1745527773 >> v2:172.21.15.155:3300/0 conn(0x7fd0dc0cfc10 0x7fd0dc0cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:18.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0e0817700 1 --2- 172.21.15.79:0/1745527773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0cc003a00 0x7fd0cc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:18.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0dbfff700 1 --2- 172.21.15.79:0/1745527773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc06b6b0 0x7fd0cc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:18.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0e1018700 1 --2- 172.21.15.79:0/1745527773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0cc004d90 0x7fd0cc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:18.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0e0817700 1 -- 172.21.15.79:0/1745527773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc06b6b0 msgr2=0x7fd0cc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:18.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0e0817700 1 --2- 172.21.15.79:0/1745527773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc06b6b0 0x7fd0cc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:18.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0e0817700 1 -- 172.21.15.79:0/1745527773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0cc004d90 msgr2=0x7fd0cc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:18.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0e0817700 1 --2- 172.21.15.79:0/1745527773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0cc004d90 0x7fd0cc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:18.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0e0817700 1 -- 172.21.15.79:0/1745527773 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc0035b0 con 0x7fd0cc003a00 2023-06-23T18:37:18.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0e1018700 1 --2- 172.21.15.79:0/1745527773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0cc004d90 0x7fd0cc005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:18.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0e0817700 1 --2- 172.21.15.79:0/1745527773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0cc003a00 0x7fd0cc003e10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000c350 tx=0x7fd0cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb424cc58e771574 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:18.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 -- 172.21.15.79:0/1745527773 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000cdf0 con 0x7fd0cc003a00 2023-06-23T18:37:18.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 -- 172.21.15.79:0/1745527773 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d0007bf0 con 0x7fd0cc003a00 2023-06-23T18:37:18.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.126+0000 7fd0db7fe700 1 -- 172.21.15.79:0/1745527773 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0007310 con 0x7fd0cc003a00 2023-06-23T18:37:18.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.127+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1745527773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0cc003a00 msgr2=0x7fd0cc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:18.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.127+0000 7fd0e2a7b700 1 --2- 172.21.15.79:0/1745527773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0cc003a00 0x7fd0cc003e10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000c350 tx=0x7fd0cc005880 comp rx=0 tx=0).stop 2023-06-23T18:37:18.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.127+0000 7fd0e1018700 1 -- 172.21.15.79:0/1745527773 reap_dead start 2023-06-23T18:37:18.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.127+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1745527773 shutdown_connections 2023-06-23T18:37:18.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.127+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1745527773 >> 172.21.15.79:0/1745527773 conn(0x7fd0dc06a7b0 msgr2=0x7fd0dc099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:18.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.127+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1745527773 shutdown_connections 2023-06-23T18:37:18.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.127+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1745527773 wait complete. 2023-06-23T18:37:18.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.128+0000 7fd0e2a7b700 1 Processor -- start 2023-06-23T18:37:18.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.129+0000 7fd0e2a7b700 1 -- start start 2023-06-23T18:37:18.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e2a7b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0cc003a00 0x7fd0dc0b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:18.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e2a7b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0dc0c6450 0x7fd0dc15a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:18.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e2a7b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc0b3dc0 0x7fd0dc0b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:18.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e2a7b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc0b47a0 con 0x7fd0cc003a00 2023-06-23T18:37:18.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e2a7b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc0caf90 con 0x7fd0dc0c6450 2023-06-23T18:37:18.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e2a7b700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc0b4c10 con 0x7fd0dc0b3dc0 2023-06-23T18:37:18.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e0817700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0cc003a00 0x7fd0dc0b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:18.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e0817700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0cc003a00 0x7fd0dc0b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42136/0 (socket says 172.21.15.79:42136) 2023-06-23T18:37:18.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e0817700 1 -- 172.21.15.79:0/1684511716 learned_addr learned my addr 172.21.15.79:0/1684511716 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:18.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0dbfff700 1 --2- 172.21.15.79:0/1684511716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0dc0c6450 0x7fd0dc15a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:18.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e1018700 1 --2- 172.21.15.79:0/1684511716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc0b3dc0 0x7fd0dc0b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:18.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e0817700 1 -- 172.21.15.79:0/1684511716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc0b3dc0 msgr2=0x7fd0dc0b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:18.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e0817700 1 --2- 172.21.15.79:0/1684511716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc0b3dc0 0x7fd0dc0b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:18.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e0817700 1 -- 172.21.15.79:0/1684511716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0dc0c6450 msgr2=0x7fd0dc15a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:18.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e0817700 1 --2- 172.21.15.79:0/1684511716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0dc0c6450 0x7fd0dc15a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:18.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e0817700 1 -- 172.21.15.79:0/1684511716 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc15a760 con 0x7fd0cc003a00 2023-06-23T18:37:18.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e0817700 1 --2- 172.21.15.79:0/1684511716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0cc003a00 0x7fd0dc0b3790 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0015040 tx=0x7fd0d0001bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:18.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0d97fa700 1 -- 172.21.15.79:0/1684511716 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0005ea0 con 0x7fd0cc003a00 2023-06-23T18:37:18.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0d97fa700 1 -- 172.21.15.79:0/1684511716 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d00072f0 con 0x7fd0cc003a00 2023-06-23T18:37:18.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1684511716 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0dc0c6a10 con 0x7fd0cc003a00 2023-06-23T18:37:18.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0d97fa700 1 -- 172.21.15.79:0/1684511716 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d00076a0 con 0x7fd0cc003a00 2023-06-23T18:37:18.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.130+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1684511716 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0dc0c6fd0 con 0x7fd0cc003a00 2023-06-23T18:37:18.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.131+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1684511716 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0dc0672c0 con 0x7fd0cc003a00 2023-06-23T18:37:18.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.132+0000 7fd0d97fa700 1 -- 172.21.15.79:0/1684511716 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd0d001e430 con 0x7fd0cc003a00 2023-06-23T18:37:18.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.132+0000 7fd0d97fa700 1 --2- 172.21.15.79:0/1684511716 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd0c006ad10 0x7fd0c006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:18.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.132+0000 7fd0d97fa700 1 -- 172.21.15.79:0/1684511716 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fd0d008a6b0 con 0x7fd0cc003a00 2023-06-23T18:37:18.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.132+0000 7fd0dbfff700 1 --2- 172.21.15.79:0/1684511716 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd0c006ad10 0x7fd0c006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:18.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.132+0000 7fd0dbfff700 1 --2- 172.21.15.79:0/1684511716 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd0c006ad10 0x7fd0c006d1c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc06cb70 tx=0x7fd0c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:18.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.136+0000 7fd0d97fa700 1 -- 172.21.15.79:0/1684511716 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd0d0055790 con 0x7fd0cc003a00 2023-06-23T18:37:18.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.365+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1684511716 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd0dc0b0dc0 con 0x7fd0cc003a00 2023-06-23T18:37:18.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.980+0000 7fd0d97fa700 1 -- 172.21.15.79:0/1684511716 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd0d0058db0 con 0x7fd0cc003a00 2023-06-23T18:37:18.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.982+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1684511716 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd0c006ad10 msgr2=0x7fd0c006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:18.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.982+0000 7fd0e2a7b700 1 --2- 172.21.15.79:0/1684511716 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd0c006ad10 0x7fd0c006d1c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc06cb70 tx=0x7fd0c8008040 comp rx=0 tx=0).stop 2023-06-23T18:37:18.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.982+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1684511716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0cc003a00 msgr2=0x7fd0dc0b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:18.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.982+0000 7fd0e2a7b700 1 --2- 172.21.15.79:0/1684511716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0cc003a00 0x7fd0dc0b3790 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0015040 tx=0x7fd0d0001bf0 comp rx=0 tx=0).stop 2023-06-23T18:37:18.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.982+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1684511716 shutdown_connections 2023-06-23T18:37:18.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.982+0000 7fd0e2a7b700 1 --2- 172.21.15.79:0/1684511716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc0b3dc0 0x7fd0dc0b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:18.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.982+0000 7fd0e2a7b700 1 --2- 172.21.15.79:0/1684511716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0cc003a00 0x7fd0dc0b3790 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:18.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.982+0000 7fd0e2a7b700 1 --2- 172.21.15.79:0/1684511716 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd0c006ad10 0x7fd0c006d1c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:18.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.982+0000 7fd0e2a7b700 1 --2- 172.21.15.79:0/1684511716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0dc0c6450 0x7fd0dc15a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:18.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.982+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1684511716 >> 172.21.15.79:0/1684511716 conn(0x7fd0dc06a7b0 msgr2=0x7fd0dc06f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:18.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.982+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1684511716 shutdown_connections 2023-06-23T18:37:18.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:18.982+0000 7fd0e2a7b700 1 -- 172.21.15.79:0/1684511716 wait complete. 2023-06-23T18:37:19.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-06-23T18:37:19.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.162+0000 7f3da437c700 1 Processor -- start 2023-06-23T18:37:19.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.162+0000 7f3da437c700 1 -- start start 2023-06-23T18:37:19.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.163+0000 7f3da437c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3d9c069e90 0x7f3d9c06a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:19.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.163+0000 7f3da437c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d9c0c95d0 0x7f3d9c0c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:19.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.163+0000 7f3da437c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d9c0c9f50 con 0x7f3d9c158910 2023-06-23T18:37:19.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.163+0000 7f3da437c700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9c0ccaa0 con 0x7f3d9c069e90 2023-06-23T18:37:19.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.163+0000 7f3da437c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c06a910 con 0x7f3d9c0c95d0 2023-06-23T18:37:19.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.163+0000 7f3da2118700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3d9c158910 0x7f3d9c06c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45188/0 (socket says 172.21.15.79:45188) 2023-06-23T18:37:19.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.163+0000 7f3da2118700 1 -- 172.21.15.79:0/2305178787 learned_addr learned my addr 172.21.15.79:0/2305178787 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:19.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.163+0000 7f3da1116700 1 -- 172.21.15.79:0/2305178787 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555007525 0 0) 0x7f3d9c0c9f50 con 0x7f3d9c158910 2023-06-23T18:37:19.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.163+0000 7f3da1116700 1 -- 172.21.15.79:0/2305178787 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d880035b0 con 0x7f3d9c158910 2023-06-23T18:37:19.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da2919700 1 --2- 172.21.15.79:0/2305178787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d9c0c95d0 0x7f3d9c0c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:19.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1917700 1 --2- 172.21.15.79:0/2305178787 >> v2:172.21.15.155:3300/0 conn(0x7f3d9c069e90 0x7f3d9c06a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:19.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1116700 1 -- 172.21.15.79:0/2305178787 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 153455255 0 0) 0x7f3d880035b0 con 0x7f3d9c158910 2023-06-23T18:37:19.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1116700 1 -- 172.21.15.79:0/2305178787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d9c0c95d0 msgr2=0x7f3d9c0c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:19.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1116700 1 --2- 172.21.15.79:0/2305178787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d9c0c95d0 0x7f3d9c0c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:19.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1116700 1 -- 172.21.15.79:0/2305178787 >> v2:172.21.15.155:3300/0 conn(0x7f3d9c069e90 msgr2=0x7f3d9c06a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:19.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1116700 1 --2- 172.21.15.79:0/2305178787 >> v2:172.21.15.155:3300/0 conn(0x7f3d9c069e90 0x7f3d9c06a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:19.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1116700 1 -- 172.21.15.79:0/2305178787 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c0cccb0 con 0x7f3d9c158910 2023-06-23T18:37:19.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1116700 1 -- 172.21.15.79:0/2305178787 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f3d8c001670 con 0x7f3d9c158910 2023-06-23T18:37:19.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1116700 1 -- 172.21.15.79:0/2305178787 >> v1:172.21.15.79:6789/0 conn(0x7f3d9c158910 legacy=0x7f3d9c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:19.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1116700 1 --2- 172.21.15.79:0/2305178787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d9c0c95d0 0x7f3d88003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:19.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1116700 1 --2- 172.21.15.79:0/2305178787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d88003f80 0x7f3d88004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:19.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1116700 1 --2- 172.21.15.79:0/2305178787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d88004d90 0x7f3d88005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:19.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1116700 1 -- 172.21.15.79:0/2305178787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0ccaa0 con 0x7f3d88003f80 2023-06-23T18:37:19.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1116700 1 -- 172.21.15.79:0/2305178787 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c06a910 con 0x7f3d88004d90 2023-06-23T18:37:19.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1116700 1 -- 172.21.15.79:0/2305178787 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8800ba10 con 0x7f3d9c0c95d0 2023-06-23T18:37:19.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1917700 1 --2- 172.21.15.79:0/2305178787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d88003f80 0x7f3d88004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:19.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da2118700 1 --2- 172.21.15.79:0/2305178787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d9c0c95d0 0x7f3d88003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:19.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da2919700 1 --2- 172.21.15.79:0/2305178787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d88004d90 0x7f3d88005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:19.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1917700 1 -- 172.21.15.79:0/2305178787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d9c0c95d0 msgr2=0x7f3d88003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:19.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1917700 1 --2- 172.21.15.79:0/2305178787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d9c0c95d0 0x7f3d88003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:19.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1917700 1 -- 172.21.15.79:0/2305178787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d88004d90 msgr2=0x7f3d88005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:19.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1917700 1 --2- 172.21.15.79:0/2305178787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d88004d90 0x7f3d88005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:19.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1917700 1 -- 172.21.15.79:0/2305178787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d880035b0 con 0x7f3d88003f80 2023-06-23T18:37:19.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da2919700 1 --2- 172.21.15.79:0/2305178787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d88004d90 0x7f3d88005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:19.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.164+0000 7f3da1917700 1 --2- 172.21.15.79:0/2305178787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d88003f80 0x7f3d88004850 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009b60 tx=0x7f3d88005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9dc4690b1d8580 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:19.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.165+0000 7f3da1116700 1 -- 172.21.15.79:0/2305178787 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9800e6c0 con 0x7f3d88003f80 2023-06-23T18:37:19.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.165+0000 7f3da1116700 1 -- 172.21.15.79:0/2305178787 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9800e820 con 0x7f3d88003f80 2023-06-23T18:37:19.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.165+0000 7f3da1116700 1 -- 172.21.15.79:0/2305178787 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9800ebb0 con 0x7f3d88003f80 2023-06-23T18:37:19.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.165+0000 7f3da437c700 1 -- 172.21.15.79:0/2305178787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d88003f80 msgr2=0x7f3d88004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:19.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.165+0000 7f3da437c700 1 --2- 172.21.15.79:0/2305178787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d88003f80 0x7f3d88004850 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009b60 tx=0x7f3d88005880 comp rx=0 tx=0).stop 2023-06-23T18:37:19.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.165+0000 7f3da2919700 1 -- 172.21.15.79:0/2305178787 reap_dead start 2023-06-23T18:37:19.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.165+0000 7f3da437c700 1 -- 172.21.15.79:0/2305178787 shutdown_connections 2023-06-23T18:37:19.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.165+0000 7f3da437c700 1 -- 172.21.15.79:0/2305178787 >> 172.21.15.79:0/2305178787 conn(0x7f3d9c0cbe10 msgr2=0x7f3d9c0ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:19.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.165+0000 7f3da437c700 1 -- 172.21.15.79:0/2305178787 shutdown_connections 2023-06-23T18:37:19.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.165+0000 7f3da437c700 1 -- 172.21.15.79:0/2305178787 wait complete. 2023-06-23T18:37:19.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.166+0000 7f3da437c700 1 Processor -- start 2023-06-23T18:37:19.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.166+0000 7f3da437c700 1 -- start start 2023-06-23T18:37:19.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.167+0000 7f3da437c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d88003f80 0x7f3d9c154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:19.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.167+0000 7f3da437c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d9c0b0720 0x7f3d9c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:19.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.167+0000 7f3da437c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9c0b0b90 0x7f3d9c0b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:19.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.167+0000 7f3da437c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0b10e0 con 0x7f3d9c0b0720 2023-06-23T18:37:19.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.167+0000 7f3da437c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0b43d0 con 0x7f3d9c0b0b90 2023-06-23T18:37:19.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.167+0000 7f3da437c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c155360 con 0x7f3d88003f80 2023-06-23T18:37:19.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.167+0000 7f3da1917700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d9c0b0720 0x7f3d9c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:19.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.167+0000 7f3da1917700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d9c0b0720 0x7f3d9c0b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42160/0 (socket says 172.21.15.79:42160) 2023-06-23T18:37:19.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.167+0000 7f3da1917700 1 -- 172.21.15.79:0/2361408897 learned_addr learned my addr 172.21.15.79:0/2361408897 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:19.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.167+0000 7f3da2118700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d88003f80 0x7f3d9c154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:19.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.167+0000 7f3da2919700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9c0b0b90 0x7f3d9c0b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:19.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.167+0000 7f3da1917700 1 -- 172.21.15.79:0/2361408897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d88003f80 msgr2=0x7f3d9c154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:19.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.168+0000 7f3da1917700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d88003f80 0x7f3d9c154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:19.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.168+0000 7f3da1917700 1 -- 172.21.15.79:0/2361408897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9c0b0b90 msgr2=0x7f3d9c0b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:19.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.168+0000 7f3da1917700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9c0b0b90 0x7f3d9c0b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:19.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.168+0000 7f3da1917700 1 -- 172.21.15.79:0/2361408897 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c0c39c0 con 0x7f3d9c0b0720 2023-06-23T18:37:19.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.168+0000 7f3da2919700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9c0b0b90 0x7f3d9c0b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:19.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.168+0000 7f3da1917700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d9c0b0720 0x7f3d9c0b3950 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3d980090b0 tx=0x7f3d980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:19.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.168+0000 7f3d92ffd700 1 -- 172.21.15.79:0/2361408897 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98006e60 con 0x7f3d9c0b0720 2023-06-23T18:37:19.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.168+0000 7f3d92ffd700 1 -- 172.21.15.79:0/2361408897 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d980038a0 con 0x7f3d9c0b0720 2023-06-23T18:37:19.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.168+0000 7f3da437c700 1 -- 172.21.15.79:0/2361408897 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9c0c3ca0 con 0x7f3d9c0b0720 2023-06-23T18:37:19.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.168+0000 7f3d92ffd700 1 -- 172.21.15.79:0/2361408897 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9800e6c0 con 0x7f3d9c0b0720 2023-06-23T18:37:19.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.168+0000 7f3da437c700 1 -- 172.21.15.79:0/2361408897 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9c158f50 con 0x7f3d9c0b0720 2023-06-23T18:37:19.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.169+0000 7f3da437c700 1 -- 172.21.15.79:0/2361408897 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9c04e1d0 con 0x7f3d9c0b0720 2023-06-23T18:37:19.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.169+0000 7f3d92ffd700 1 -- 172.21.15.79:0/2361408897 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3d98020040 con 0x7f3d9c0b0720 2023-06-23T18:37:19.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.170+0000 7f3d92ffd700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3d8006ad10 0x7f3d8006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:19.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.170+0000 7f3d92ffd700 1 -- 172.21.15.79:0/2361408897 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f3d9808bf20 con 0x7f3d9c0b0720 2023-06-23T18:37:19.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.170+0000 7f3da2118700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3d8006ad10 0x7f3d8006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:19.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.170+0000 7f3da2118700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3d8006ad10 0x7f3d8006d1c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c000fb0 tx=0x7f3d8c007980 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:19.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.174+0000 7f3d92ffd700 1 -- 172.21.15.79:0/2361408897 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3d9805ab10 con 0x7f3d9c0b0720 2023-06-23T18:37:19.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.362+0000 7f3da437c700 1 -- 172.21.15.79:0/2361408897 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3d9c0c8fe0 con 0x7f3d9c0b0720 2023-06-23T18:37:19.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.986+0000 7f3d92ffd700 1 -- 172.21.15.79:0/2361408897 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3d9805a6a0 con 0x7f3d9c0b0720 2023-06-23T18:37:19.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.989+0000 7f3da437c700 1 -- 172.21.15.79:0/2361408897 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3d8006ad10 msgr2=0x7f3d8006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:19.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.989+0000 7f3da437c700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3d8006ad10 0x7f3d8006d1c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c000fb0 tx=0x7f3d8c007980 comp rx=0 tx=0).stop 2023-06-23T18:37:19.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.989+0000 7f3da437c700 1 -- 172.21.15.79:0/2361408897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d9c0b0720 msgr2=0x7f3d9c0b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:19.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.989+0000 7f3da437c700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d9c0b0720 0x7f3d9c0b3950 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3d980090b0 tx=0x7f3d980047c0 comp rx=0 tx=0).stop 2023-06-23T18:37:19.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.989+0000 7f3da437c700 1 -- 172.21.15.79:0/2361408897 shutdown_connections 2023-06-23T18:37:19.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.989+0000 7f3da437c700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f3d8006ad10 0x7f3d8006d1c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:19.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.989+0000 7f3da437c700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9c0b0b90 0x7f3d9c0b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:19.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.989+0000 7f3da437c700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d88003f80 0x7f3d9c154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:19.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.989+0000 7f3da437c700 1 --2- 172.21.15.79:0/2361408897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d9c0b0720 0x7f3d9c0b3950 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:19.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.989+0000 7f3da437c700 1 -- 172.21.15.79:0/2361408897 >> 172.21.15.79:0/2361408897 conn(0x7f3d9c0cbe10 msgr2=0x7f3d9c0c59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:19.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.989+0000 7f3da437c700 1 -- 172.21.15.79:0/2361408897 shutdown_connections 2023-06-23T18:37:19.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:19.989+0000 7f3da437c700 1 -- 172.21.15.79:0/2361408897 wait complete. 2023-06-23T18:37:19.996 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_metadata' removed 2023-06-23T18:37:20.008 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-06-23T18:37:20.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.130+0000 7f1a52b25700 1 Processor -- start 2023-06-23T18:37:20.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.130+0000 7f1a52b25700 1 -- start start 2023-06-23T18:37:20.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.130+0000 7f1a52b25700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1a4c15a130 0x7f1a4c15b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:20.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.130+0000 7f1a52b25700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1a4c071780 0x7f1a4c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:20.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.130+0000 7f1a52b25700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4c0c98d0 con 0x7f1a4c06cc30 2023-06-23T18:37:20.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.130+0000 7f1a52b25700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a4c15b550 con 0x7f1a4c15a130 2023-06-23T18:37:20.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.130+0000 7f1a52b25700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a4c071c80 con 0x7f1a4c071780 2023-06-23T18:37:20.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.131+0000 7f1a508c1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1a4c15a130 0x7f1a4c15b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:20.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.131+0000 7f1a508c1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1a4c15a130 0x7f1a4c15b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42166/0 (socket says 172.21.15.79:42166) 2023-06-23T18:37:20.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.131+0000 7f1a508c1700 1 -- 172.21.15.79:0/119902807 learned_addr learned my addr 172.21.15.79:0/119902807 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:20.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.131+0000 7f1a4bfff700 1 --2- 172.21.15.79:0/119902807 >> v2:172.21.15.155:3300/0 conn(0x7f1a4c071780 0x7f1a4c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:20.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.131+0000 7f1a508c1700 1 -- 172.21.15.79:0/119902807 >> v2:172.21.15.155:3300/0 conn(0x7f1a4c071780 msgr2=0x7f1a4c0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:20.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.131+0000 7f1a508c1700 1 --2- 172.21.15.79:0/119902807 >> v2:172.21.15.155:3300/0 conn(0x7f1a4c071780 0x7f1a4c0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:20.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.131+0000 7f1a508c1700 1 -- 172.21.15.79:0/119902807 >> v1:172.21.15.155:6789/0 conn(0x7f1a4c06cc30 legacy=0x7f1a4c15ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:20.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.131+0000 7f1a508c1700 1 -- 172.21.15.79:0/119902807 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c06b250 con 0x7f1a4c15a130 2023-06-23T18:37:20.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.131+0000 7f1a508c1700 1 --2- 172.21.15.79:0/119902807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a4c15a130 0x7f1a4c15b010 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c071540 tx=0x7f1a40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173782a41de608df server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:20.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.131+0000 7f1a4b7fe700 1 -- 172.21.15.79:0/119902807 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40004230 con 0x7f1a4c15a130 2023-06-23T18:37:20.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.131+0000 7f1a4b7fe700 1 -- 172.21.15.79:0/119902807 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a40004390 con 0x7f1a4c15a130 2023-06-23T18:37:20.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.131+0000 7f1a4b7fe700 1 -- 172.21.15.79:0/119902807 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a400036a0 con 0x7f1a4c15a130 2023-06-23T18:37:20.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.132+0000 7f1a52b25700 1 -- 172.21.15.79:0/119902807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a4c15a130 msgr2=0x7f1a4c15b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:20.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.132+0000 7f1a52b25700 1 --2- 172.21.15.79:0/119902807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a4c15a130 0x7f1a4c15b010 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c071540 tx=0x7f1a40009bd0 comp rx=0 tx=0).stop 2023-06-23T18:37:20.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.132+0000 7f1a52b25700 1 -- 172.21.15.79:0/119902807 shutdown_connections 2023-06-23T18:37:20.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.132+0000 7f1a52b25700 1 --2- 172.21.15.79:0/119902807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a4c15a130 0x7f1a4c15b010 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:20.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.132+0000 7f1a52b25700 1 --2- 172.21.15.79:0/119902807 >> v2:172.21.15.155:3300/0 conn(0x7f1a4c071780 0x7f1a4c0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:20.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.132+0000 7f1a52b25700 1 -- 172.21.15.79:0/119902807 >> 172.21.15.79:0/119902807 conn(0x7f1a4c0cefc0 msgr2=0x7f1a4c0cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:20.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.132+0000 7f1a52b25700 1 -- 172.21.15.79:0/119902807 shutdown_connections 2023-06-23T18:37:20.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.132+0000 7f1a52b25700 1 -- 172.21.15.79:0/119902807 wait complete. 2023-06-23T18:37:20.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.133+0000 7f1a52b25700 1 Processor -- start 2023-06-23T18:37:20.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.133+0000 7f1a52b25700 1 -- start start 2023-06-23T18:37:20.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a52b25700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a4c06cc30 0x7f1a4c1f0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:20.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a52b25700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a4c071780 0x7f1a4c0b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:20.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a52b25700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a4c15a130 0x7f1a4c0b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:20.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a52b25700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0b0e00 con 0x7f1a4c15a130 2023-06-23T18:37:20.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a52b25700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0afab0 con 0x7f1a4c071780 2023-06-23T18:37:20.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a52b25700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0b04d0 con 0x7f1a4c06cc30 2023-06-23T18:37:20.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a510c2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a4c15a130 0x7f1a4c0b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:20.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a510c2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a4c15a130 0x7f1a4c0b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42174/0 (socket says 172.21.15.79:42174) 2023-06-23T18:37:20.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a510c2700 1 -- 172.21.15.79:0/3444806960 learned_addr learned my addr 172.21.15.79:0/3444806960 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:20.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a4bfff700 1 --2- 172.21.15.79:0/3444806960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a4c071780 0x7f1a4c0b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:20.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a508c1700 1 --2- 172.21.15.79:0/3444806960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a4c06cc30 0x7f1a4c1f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:20.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a510c2700 1 -- 172.21.15.79:0/3444806960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a4c06cc30 msgr2=0x7f1a4c1f0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:20.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a510c2700 1 --2- 172.21.15.79:0/3444806960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a4c06cc30 0x7f1a4c1f0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:20.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a510c2700 1 -- 172.21.15.79:0/3444806960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a4c071780 msgr2=0x7f1a4c0b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:20.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a510c2700 1 --2- 172.21.15.79:0/3444806960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a4c071780 0x7f1a4c0b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:20.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a510c2700 1 -- 172.21.15.79:0/3444806960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a40009550 con 0x7f1a4c15a130 2023-06-23T18:37:20.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a510c2700 1 --2- 172.21.15.79:0/3444806960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a4c15a130 0x7f1a4c0b4390 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400cc10 tx=0x7f1a4400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:20.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.134+0000 7f1a497fa700 1 -- 172.21.15.79:0/3444806960 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4400ab10 con 0x7f1a4c15a130 2023-06-23T18:37:20.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.135+0000 7f1a497fa700 1 -- 172.21.15.79:0/3444806960 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4400ac70 con 0x7f1a4c15a130 2023-06-23T18:37:20.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.135+0000 7f1a52b25700 1 -- 172.21.15.79:0/3444806960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4c0b07b0 con 0x7f1a4c15a130 2023-06-23T18:37:20.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.135+0000 7f1a497fa700 1 -- 172.21.15.79:0/3444806960 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44012420 con 0x7f1a4c15a130 2023-06-23T18:37:20.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.135+0000 7f1a52b25700 1 -- 172.21.15.79:0/3444806960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c1f0850 con 0x7f1a4c15a130 2023-06-23T18:37:20.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.135+0000 7f1a52b25700 1 -- 172.21.15.79:0/3444806960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4c0672c0 con 0x7f1a4c15a130 2023-06-23T18:37:20.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.136+0000 7f1a497fa700 1 -- 172.21.15.79:0/3444806960 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1a44012580 con 0x7f1a4c15a130 2023-06-23T18:37:20.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.136+0000 7f1a497fa700 1 --2- 172.21.15.79:0/3444806960 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1a3006ace0 0x7f1a3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:20.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.136+0000 7f1a508c1700 1 --2- 172.21.15.79:0/3444806960 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1a3006ace0 0x7f1a3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:20.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.136+0000 7f1a497fa700 1 -- 172.21.15.79:0/3444806960 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f1a44089650 con 0x7f1a4c15a130 2023-06-23T18:37:20.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.137+0000 7f1a508c1700 1 --2- 172.21.15.79:0/3444806960 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1a3006ace0 0x7f1a3006d190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009130 tx=0x7f1a40008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:20.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.141+0000 7f1a497fa700 1 -- 172.21.15.79:0/3444806960 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1a44058380 con 0x7f1a4c15a130 2023-06-23T18:37:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.328+0000 7f1a52b25700 1 -- 172.21.15.79:0/3444806960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1a4c0cbc00 con 0x7f1a4c15a130 2023-06-23T18:37:20.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.990+0000 7f1a497fa700 1 -- 172.21.15.79:0/3444806960 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1a44057f10 con 0x7f1a4c15a130 2023-06-23T18:37:20.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.993+0000 7f1a52b25700 1 -- 172.21.15.79:0/3444806960 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1a3006ace0 msgr2=0x7f1a3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:20.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.993+0000 7f1a52b25700 1 --2- 172.21.15.79:0/3444806960 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1a3006ace0 0x7f1a3006d190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009130 tx=0x7f1a40008030 comp rx=0 tx=0).stop 2023-06-23T18:37:20.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.993+0000 7f1a52b25700 1 -- 172.21.15.79:0/3444806960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a4c15a130 msgr2=0x7f1a4c0b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:20.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.993+0000 7f1a52b25700 1 --2- 172.21.15.79:0/3444806960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a4c15a130 0x7f1a4c0b4390 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400cc10 tx=0x7f1a4400cf20 comp rx=0 tx=0).stop 2023-06-23T18:37:20.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.993+0000 7f1a52b25700 1 -- 172.21.15.79:0/3444806960 shutdown_connections 2023-06-23T18:37:20.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.993+0000 7f1a52b25700 1 --2- 172.21.15.79:0/3444806960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a4c15a130 0x7f1a4c0b4390 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.993+0000 7f1a52b25700 1 --2- 172.21.15.79:0/3444806960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a4c06cc30 0x7f1a4c1f0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.993+0000 7f1a52b25700 1 --2- 172.21.15.79:0/3444806960 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1a3006ace0 0x7f1a3006d190 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.993+0000 7f1a52b25700 1 --2- 172.21.15.79:0/3444806960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a4c071780 0x7f1a4c0b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.993+0000 7f1a52b25700 1 -- 172.21.15.79:0/3444806960 >> 172.21.15.79:0/3444806960 conn(0x7f1a4c0cefc0 msgr2=0x7f1a4c15d010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:21.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.993+0000 7f1a52b25700 1 -- 172.21.15.79:0/3444806960 shutdown_connections 2023-06-23T18:37:21.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:20.994+0000 7f1a52b25700 1 -- 172.21.15.79:0/3444806960 wait complete. 2023-06-23T18:37:21.001 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data' removed 2023-06-23T18:37:21.011 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-06-23T18:37:21.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.128+0000 7fdf5b5c8700 1 Processor -- start 2023-06-23T18:37:21.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.129+0000 7fdf5b5c8700 1 -- start start 2023-06-23T18:37:21.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf5b5c8700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdf540cc060 0x7fdf540cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf5b5c8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdf54071710 0x7fdf54071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf5b5c8700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf540cc9f0 con 0x7fdf54156ae0 2023-06-23T18:37:21.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf5b5c8700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5415a560 con 0x7fdf54071710 2023-06-23T18:37:21.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf5b5c8700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf54159950 con 0x7fdf540cc060 2023-06-23T18:37:21.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf59b65700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdf54071710 0x7fdf54071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf59364700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fdf54156ae0 0x7fdf5415d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45220/0 (socket says 172.21.15.79:45220) 2023-06-23T18:37:21.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf59364700 1 -- 172.21.15.79:0/1292458367 learned_addr learned my addr 172.21.15.79:0/1292458367 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:21.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf58b63700 1 --2- 172.21.15.79:0/1292458367 >> v2:172.21.15.155:3300/0 conn(0x7fdf540cc060 0x7fdf540cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf59b65700 1 -- 172.21.15.79:0/1292458367 >> v2:172.21.15.155:3300/0 conn(0x7fdf540cc060 msgr2=0x7fdf540cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:21.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf59b65700 1 --2- 172.21.15.79:0/1292458367 >> v2:172.21.15.155:3300/0 conn(0x7fdf540cc060 0x7fdf540cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf59b65700 1 -- 172.21.15.79:0/1292458367 >> v1:172.21.15.79:6789/0 conn(0x7fdf54156ae0 legacy=0x7fdf5415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf4bfff700 1 -- 172.21.15.79:0/1292458367 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220042174 0 0) 0x7fdf44001110 con 0x7fdf54156ae0 2023-06-23T18:37:21.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf59b65700 1 -- 172.21.15.79:0/1292458367 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf54159c30 con 0x7fdf54071710 2023-06-23T18:37:21.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf59b65700 1 --2- 172.21.15.79:0/1292458367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf54071710 0x7fdf54071b50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdf54070f20 tx=0x7fdf4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0ba318efd54b6c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:21.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf4bfff700 1 -- 172.21.15.79:0/1292458367 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c00e070 con 0x7fdf54071710 2023-06-23T18:37:21.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf4bfff700 1 -- 172.21.15.79:0/1292458367 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c00bbd0 con 0x7fdf54071710 2023-06-23T18:37:21.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.130+0000 7fdf4bfff700 1 -- 172.21.15.79:0/1292458367 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c012430 con 0x7fdf54071710 2023-06-23T18:37:21.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.131+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/1292458367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf54071710 msgr2=0x7fdf54071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.131+0000 7fdf5b5c8700 1 --2- 172.21.15.79:0/1292458367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf54071710 0x7fdf54071b50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdf54070f20 tx=0x7fdf4c00b330 comp rx=0 tx=0).stop 2023-06-23T18:37:21.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.131+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/1292458367 shutdown_connections 2023-06-23T18:37:21.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.131+0000 7fdf5b5c8700 1 --2- 172.21.15.79:0/1292458367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf54071710 0x7fdf54071b50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.131+0000 7fdf5b5c8700 1 --2- 172.21.15.79:0/1292458367 >> v2:172.21.15.155:3300/0 conn(0x7fdf540cc060 0x7fdf540cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.131+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/1292458367 >> 172.21.15.79:0/1292458367 conn(0x7fdf5406afa0 msgr2=0x7fdf5406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:21.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.131+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/1292458367 shutdown_connections 2023-06-23T18:37:21.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.131+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/1292458367 wait complete. 2023-06-23T18:37:21.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.132+0000 7fdf5b5c8700 1 Processor -- start 2023-06-23T18:37:21.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.132+0000 7fdf5b5c8700 1 -- start start 2023-06-23T18:37:21.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf5b5c8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf54071710 0x7fdf5414bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf5b5c8700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf540cc060 0x7fdf540c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf5b5c8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf54156ae0 0x7fdf540c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf5b5c8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf541ef010 con 0x7fdf54071710 2023-06-23T18:37:21.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf5b5c8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf541ef2f0 con 0x7fdf54156ae0 2023-06-23T18:37:21.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf5b5c8700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf541ef5d0 con 0x7fdf540cc060 2023-06-23T18:37:21.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf59364700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf54071710 0x7fdf5414bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf59364700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf54071710 0x7fdf5414bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42184/0 (socket says 172.21.15.79:42184) 2023-06-23T18:37:21.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf59364700 1 -- 172.21.15.79:0/2715811508 learned_addr learned my addr 172.21.15.79:0/2715811508 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:21.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf59b65700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf54156ae0 0x7fdf540c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf58b63700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf540cc060 0x7fdf540c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf59364700 1 -- 172.21.15.79:0/2715811508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf540cc060 msgr2=0x7fdf540c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf59364700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf540cc060 0x7fdf540c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf59364700 1 -- 172.21.15.79:0/2715811508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf54156ae0 msgr2=0x7fdf540c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf59364700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf54156ae0 0x7fdf540c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf59364700 1 -- 172.21.15.79:0/2715811508 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c009a70 con 0x7fdf54071710 2023-06-23T18:37:21.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf59b65700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf54156ae0 0x7fdf540c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:21.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf58b63700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf540cc060 0x7fdf540c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:21.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf59364700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf54071710 0x7fdf5414bd60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400c550 tx=0x7fdf4400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:21.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf49ffb700 1 -- 172.21.15.79:0/2715811508 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44004c80 con 0x7fdf54071710 2023-06-23T18:37:21.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf49ffb700 1 -- 172.21.15.79:0/2715811508 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf44004e00 con 0x7fdf54071710 2023-06-23T18:37:21.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/2715811508 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf541ef8b0 con 0x7fdf54071710 2023-06-23T18:37:21.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.133+0000 7fdf49ffb700 1 -- 172.21.15.79:0/2715811508 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44007310 con 0x7fdf54071710 2023-06-23T18:37:21.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.134+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/2715811508 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf541eecc0 con 0x7fdf54071710 2023-06-23T18:37:21.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.135+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/2715811508 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf541ef150 con 0x7fdf54071710 2023-06-23T18:37:21.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.135+0000 7fdf49ffb700 1 -- 172.21.15.79:0/2715811508 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdf44007470 con 0x7fdf54071710 2023-06-23T18:37:21.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.135+0000 7fdf49ffb700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdf3806ace0 0x7fdf3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.135+0000 7fdf49ffb700 1 -- 172.21.15.79:0/2715811508 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdf44088d70 con 0x7fdf54071710 2023-06-23T18:37:21.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.135+0000 7fdf58b63700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdf3806ace0 0x7fdf3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.136+0000 7fdf58b63700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdf3806ace0 0x7fdf3806d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdf50006fd0 tx=0x7fdf50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:21.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.140+0000 7fdf49ffb700 1 -- 172.21.15.79:0/2715811508 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdf541ef150 con 0x7fdf54071710 2023-06-23T18:37:21.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.328+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/2715811508 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdf540cc9f0 con 0x7fdf54071710 2023-06-23T18:37:21.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.332+0000 7fdf49ffb700 1 -- 172.21.15.79:0/2715811508 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdf540cc9f0 con 0x7fdf54071710 2023-06-23T18:37:21.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.334+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/2715811508 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdf3806ace0 msgr2=0x7fdf3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.334+0000 7fdf5b5c8700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdf3806ace0 0x7fdf3806d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdf50006fd0 tx=0x7fdf50008040 comp rx=0 tx=0).stop 2023-06-23T18:37:21.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.334+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/2715811508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf54071710 msgr2=0x7fdf5414bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.334+0000 7fdf5b5c8700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf54071710 0x7fdf5414bd60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400c550 tx=0x7fdf4400c910 comp rx=0 tx=0).stop 2023-06-23T18:37:21.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.334+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/2715811508 shutdown_connections 2023-06-23T18:37:21.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.334+0000 7fdf5b5c8700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fdf3806ace0 0x7fdf3806d190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.334+0000 7fdf5b5c8700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf54156ae0 0x7fdf540c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.334+0000 7fdf5b5c8700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf54071710 0x7fdf5414bd60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.334+0000 7fdf5b5c8700 1 --2- 172.21.15.79:0/2715811508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf540cc060 0x7fdf540c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.334+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/2715811508 >> 172.21.15.79:0/2715811508 conn(0x7fdf5406afa0 msgr2=0x7fdf5415a750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:21.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.334+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/2715811508 shutdown_connections 2023-06-23T18:37:21.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.334+0000 7fdf5b5c8700 1 -- 172.21.15.79:0/2715811508 wait complete. 2023-06-23T18:37:21.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values' 2023-06-23T18:37:21.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.471+0000 7fd1b2c32700 1 Processor -- start 2023-06-23T18:37:21.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.472+0000 7fd1b2c32700 1 -- start start 2023-06-23T18:37:21.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.472+0000 7fd1b2c32700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1ac158620 0x7fd1ac06d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.472+0000 7fd1b2c32700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac06b830 con 0x7fd1ac06b3c0 2023-06-23T18:37:21.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.472+0000 7fd1b2c32700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac15d130 con 0x7fd1ac0cd6e0 2023-06-23T18:37:21.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.472+0000 7fd1b2c32700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac15c370 con 0x7fd1ac158620 2023-06-23T18:37:21.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.472+0000 7fd1abfff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd1ac06b3c0 0x7fd1ac0ccfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45236/0 (socket says 172.21.15.79:45236) 2023-06-23T18:37:21.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1abfff700 1 -- 172.21.15.79:0/2087575029 learned_addr learned my addr 172.21.15.79:0/2087575029 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:21.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 -- 172.21.15.79:0/2087575029 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483072693 0 0) 0x7fd198001110 con 0x7fd1ac06b3c0 2023-06-23T18:37:21.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1b09ce700 1 --2- 172.21.15.79:0/2087575029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1ac158620 0x7fd1ac06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 -- 172.21.15.79:0/2087575029 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd19c0035b0 con 0x7fd1ac06b3c0 2023-06-23T18:37:21.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 -- 172.21.15.79:0/2087575029 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2635165404 0 0) 0x7fd19c0035b0 con 0x7fd1ac06b3c0 2023-06-23T18:37:21.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 -- 172.21.15.79:0/2087575029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1ac158620 msgr2=0x7fd1ac06d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:21.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 --2- 172.21.15.79:0/2087575029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1ac158620 0x7fd1ac06d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 -- 172.21.15.79:0/2087575029 >> v1:172.21.15.155:6789/0 conn(0x7fd1ac0cd6e0 legacy=0x7fd1ac069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 -- 172.21.15.79:0/2087575029 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac15c650 con 0x7fd1ac06b3c0 2023-06-23T18:37:21.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 -- 172.21.15.79:0/2087575029 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7fd198001800 con 0x7fd1ac06b3c0 2023-06-23T18:37:21.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 -- 172.21.15.79:0/2087575029 >> v1:172.21.15.79:6789/0 conn(0x7fd1ac06b3c0 legacy=0x7fd1ac0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 --2- 172.21.15.79:0/2087575029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd19c003a30 0x7fd19c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 --2- 172.21.15.79:0/2087575029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd19c004850 0x7fd19c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 --2- 172.21.15.79:0/2087575029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1ac158620 0x7fd19c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 -- 172.21.15.79:0/2087575029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac15c370 con 0x7fd19c004850 2023-06-23T18:37:21.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 -- 172.21.15.79:0/2087575029 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c00dcd0 con 0x7fd19c003a30 2023-06-23T18:37:21.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1ab7fe700 1 -- 172.21.15.79:0/2087575029 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c00de60 con 0x7fd1ac158620 2023-06-23T18:37:21.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1abfff700 1 --2- 172.21.15.79:0/2087575029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd19c004850 0x7fd19c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1b09ce700 1 --2- 172.21.15.79:0/2087575029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd19c003a30 0x7fd19c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1b11cf700 1 --2- 172.21.15.79:0/2087575029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1ac158620 0x7fd19c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1abfff700 1 -- 172.21.15.79:0/2087575029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1ac158620 msgr2=0x7fd19c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:21.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1abfff700 1 --2- 172.21.15.79:0/2087575029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1ac158620 0x7fd19c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1abfff700 1 -- 172.21.15.79:0/2087575029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd19c003a30 msgr2=0x7fd19c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:21.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1abfff700 1 --2- 172.21.15.79:0/2087575029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd19c003a30 0x7fd19c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1abfff700 1 -- 172.21.15.79:0/2087575029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c0035b0 con 0x7fd19c004850 2023-06-23T18:37:21.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.473+0000 7fd1b09ce700 1 --2- 172.21.15.79:0/2087575029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd19c003a30 0x7fd19c003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:21.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.474+0000 7fd1abfff700 1 --2- 172.21.15.79:0/2087575029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd19c004850 0x7fd19c004cc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c210 tx=0x7fd19c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6defca3b7bcea0b6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:21.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.474+0000 7fd1ab7fe700 1 -- 172.21.15.79:0/2087575029 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd198007db0 con 0x7fd19c004850 2023-06-23T18:37:21.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.474+0000 7fd1ab7fe700 1 -- 172.21.15.79:0/2087575029 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19800cdb0 con 0x7fd19c004850 2023-06-23T18:37:21.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.474+0000 7fd1ab7fe700 1 -- 172.21.15.79:0/2087575029 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd198005b80 con 0x7fd19c004850 2023-06-23T18:37:21.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.474+0000 7fd1b2c32700 1 -- 172.21.15.79:0/2087575029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd19c004850 msgr2=0x7fd19c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.474+0000 7fd1b2c32700 1 --2- 172.21.15.79:0/2087575029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd19c004850 0x7fd19c004cc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c210 tx=0x7fd19c005880 comp rx=0 tx=0).stop 2023-06-23T18:37:21.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.474+0000 7fd1b11cf700 1 -- 172.21.15.79:0/2087575029 reap_dead start 2023-06-23T18:37:21.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.474+0000 7fd1b2c32700 1 -- 172.21.15.79:0/2087575029 shutdown_connections 2023-06-23T18:37:21.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.474+0000 7fd1b2c32700 1 -- 172.21.15.79:0/2087575029 >> 172.21.15.79:0/2087575029 conn(0x7fd1ac0b17e0 msgr2=0x7fd1ac06cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:21.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.474+0000 7fd1b2c32700 1 -- 172.21.15.79:0/2087575029 shutdown_connections 2023-06-23T18:37:21.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.474+0000 7fd1b2c32700 1 -- 172.21.15.79:0/2087575029 wait complete. 2023-06-23T18:37:21.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.476+0000 7fd1b2c32700 1 Processor -- start 2023-06-23T18:37:21.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.476+0000 7fd1b2c32700 1 -- start start 2023-06-23T18:37:21.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b2c32700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd19c004850 0x7fd1ac153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b2c32700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1ac156050 0x7fd1ac0c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b2c32700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1ac0afdb0 0x7fd1ac0b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b2c32700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0b0770 con 0x7fd19c004850 2023-06-23T18:37:21.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b2c32700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac159e80 con 0x7fd1ac0afdb0 2023-06-23T18:37:21.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b2c32700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac1570d0 con 0x7fd1ac156050 2023-06-23T18:37:21.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b09ce700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd19c004850 0x7fd1ac153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b09ce700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd19c004850 0x7fd1ac153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42204/0 (socket says 172.21.15.79:42204) 2023-06-23T18:37:21.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b09ce700 1 -- 172.21.15.79:0/3409782231 learned_addr learned my addr 172.21.15.79:0/3409782231 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:21.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1abfff700 1 --2- 172.21.15.79:0/3409782231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1ac156050 0x7fd1ac0c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b11cf700 1 --2- 172.21.15.79:0/3409782231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1ac0afdb0 0x7fd1ac0b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b09ce700 1 -- 172.21.15.79:0/3409782231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1ac156050 msgr2=0x7fd1ac0c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b09ce700 1 --2- 172.21.15.79:0/3409782231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1ac156050 0x7fd1ac0c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b09ce700 1 -- 172.21.15.79:0/3409782231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1ac0afdb0 msgr2=0x7fd1ac0b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b09ce700 1 --2- 172.21.15.79:0/3409782231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1ac0afdb0 0x7fd1ac0b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b09ce700 1 -- 172.21.15.79:0/3409782231 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac156550 con 0x7fd19c004850 2023-06-23T18:37:21.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.477+0000 7fd1b09ce700 1 --2- 172.21.15.79:0/3409782231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd19c004850 0x7fd1ac153550 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00118d0 tx=0x7fd1a0011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:21.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.478+0000 7fd1a97fa700 1 -- 172.21.15.79:0/3409782231 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0004220 con 0x7fd19c004850 2023-06-23T18:37:21.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.478+0000 7fd1a97fa700 1 -- 172.21.15.79:0/3409782231 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a0004380 con 0x7fd19c004850 2023-06-23T18:37:21.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.478+0000 7fd1b2c32700 1 -- 172.21.15.79:0/3409782231 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ac156830 con 0x7fd19c004850 2023-06-23T18:37:21.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.478+0000 7fd1a97fa700 1 -- 172.21.15.79:0/3409782231 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a00036a0 con 0x7fd19c004850 2023-06-23T18:37:21.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.478+0000 7fd1b2c32700 1 -- 172.21.15.79:0/3409782231 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ac156df0 con 0x7fd19c004850 2023-06-23T18:37:21.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.479+0000 7fd1b2c32700 1 -- 172.21.15.79:0/3409782231 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ac153a90 con 0x7fd19c004850 2023-06-23T18:37:21.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.479+0000 7fd1a97fa700 1 -- 172.21.15.79:0/3409782231 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd1a0003800 con 0x7fd19c004850 2023-06-23T18:37:21.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.479+0000 7fd1a97fa700 1 --2- 172.21.15.79:0/3409782231 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd19006ace0 0x7fd19006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.479+0000 7fd1a97fa700 1 -- 172.21.15.79:0/3409782231 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd1a0089670 con 0x7fd19c004850 2023-06-23T18:37:21.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.479+0000 7fd1abfff700 1 --2- 172.21.15.79:0/3409782231 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd19006ace0 0x7fd19006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.480+0000 7fd1abfff700 1 --2- 172.21.15.79:0/3409782231 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd19006ace0 0x7fd19006d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd198015040 tx=0x7fd198007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:21.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.484+0000 7fd1a97fa700 1 -- 172.21.15.79:0/3409782231 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd1a0054a20 con 0x7fd19c004850 2023-06-23T18:37:21.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.748+0000 7fd1b2c32700 1 -- 172.21.15.79:0/3409782231 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]} v 0) v1 -- 0x7fd1ac1e2cb0 con 0x7fd19c004850 2023-06-23T18:37:21.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.990+0000 7fd1a97fa700 1 -- 172.21.15.79:0/3409782231 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]}]=0 v99) v1 ==== 160+0+0 (secure 0 0 0) 0x7fd1a0058040 con 0x7fd19c004850 2023-06-23T18:37:21.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.992+0000 7fd1b2c32700 1 -- 172.21.15.79:0/3409782231 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd19006ace0 msgr2=0x7fd19006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.992+0000 7fd1b2c32700 1 --2- 172.21.15.79:0/3409782231 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd19006ace0 0x7fd19006d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd198015040 tx=0x7fd198007000 comp rx=0 tx=0).stop 2023-06-23T18:37:21.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.992+0000 7fd1b2c32700 1 -- 172.21.15.79:0/3409782231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd19c004850 msgr2=0x7fd1ac153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.992+0000 7fd1b2c32700 1 --2- 172.21.15.79:0/3409782231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd19c004850 0x7fd1ac153550 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00118d0 tx=0x7fd1a0011c90 comp rx=0 tx=0).stop 2023-06-23T18:37:21.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.992+0000 7fd1b2c32700 1 -- 172.21.15.79:0/3409782231 shutdown_connections 2023-06-23T18:37:21.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.992+0000 7fd1b2c32700 1 --2- 172.21.15.79:0/3409782231 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd19006ace0 0x7fd19006d190 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.992+0000 7fd1b2c32700 1 --2- 172.21.15.79:0/3409782231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1ac0afdb0 0x7fd1ac0b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.992+0000 7fd1b2c32700 1 --2- 172.21.15.79:0/3409782231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd19c004850 0x7fd1ac153550 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.992+0000 7fd1b2c32700 1 --2- 172.21.15.79:0/3409782231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1ac156050 0x7fd1ac0c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.992+0000 7fd1b2c32700 1 -- 172.21.15.79:0/3409782231 >> 172.21.15.79:0/3409782231 conn(0x7fd1ac0b17e0 msgr2=0x7fd1ac157930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:21.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.992+0000 7fd1b2c32700 1 -- 172.21.15.79:0/3409782231 shutdown_connections 2023-06-23T18:37:21.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:21.992+0000 7fd1b2c32700 1 -- 172.21.15.79:0/3409782231 wait complete. 2023-06-23T18:37:22.009 INFO:tasks.cephfs_test_runner:test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) ... ERROR 2023-06-23T18:37:22.010 INFO:tasks.cephfs_test_runner: 2023-06-23T18:37:22.010 INFO:tasks.cephfs_test_runner:====================================================================== 2023-06-23T18:37:22.010 INFO:tasks.cephfs_test_runner:ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-06-23T18:37:22.010 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-06-23T18:37:22.010 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-06-23T18:37:22.011 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/cephfs/test_quota.py", line 157, in test_disable_enable_human_readable_quota_values 2023-06-23T18:37:22.011 INFO:tasks.cephfs_test_runner: self.mount_a.setfattr("./subdir", "ceph.quota.max_bytes", "1K") 2023-06-23T18:37:22.011 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/cephfs/mount.py", line 1488, in setfattr 2023-06-23T18:37:22.011 INFO:tasks.cephfs_test_runner: self.run_shell(**kwargs) 2023-06-23T18:37:22.011 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-06-23T18:37:22.012 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-06-23T18:37:22.012 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run 2023-06-23T18:37:22.012 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-06-23T18:37:22.012 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run 2023-06-23T18:37:22.012 INFO:tasks.cephfs_test_runner: r.wait() 2023-06-23T18:37:22.013 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait 2023-06-23T18:37:22.013 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-06-23T18:37:22.013 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-06-23T18:37:22.013 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-06-23T18:37:22.013 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi079 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir)' 2023-06-23T18:37:22.013 INFO:tasks.cephfs_test_runner: 2023-06-23T18:37:22.014 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-06-23T18:37:22.014 INFO:tasks.cephfs_test_runner:Ran 1 test in 56.868s 2023-06-23T18:37:22.014 INFO:tasks.cephfs_test_runner: 2023-06-23T18:37:22.014 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-06-23T18:37:22.014 INFO:tasks.cephfs_test_runner: 2023-06-23T18:37:22.015 INFO:tasks.cephfs_test_runner:====================================================================== 2023-06-23T18:37:22.015 INFO:tasks.cephfs_test_runner:ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-06-23T18:37:22.015 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-06-23T18:37:22.015 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-06-23T18:37:22.015 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/cephfs/test_quota.py", line 157, in test_disable_enable_human_readable_quota_values 2023-06-23T18:37:22.016 INFO:tasks.cephfs_test_runner: self.mount_a.setfattr("./subdir", "ceph.quota.max_bytes", "1K") 2023-06-23T18:37:22.016 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/cephfs/mount.py", line 1488, in setfattr 2023-06-23T18:37:22.016 INFO:tasks.cephfs_test_runner: self.run_shell(**kwargs) 2023-06-23T18:37:22.016 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-06-23T18:37:22.016 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-06-23T18:37:22.017 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run 2023-06-23T18:37:22.017 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-06-23T18:37:22.017 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run 2023-06-23T18:37:22.017 INFO:tasks.cephfs_test_runner: r.wait() 2023-06-23T18:37:22.017 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait 2023-06-23T18:37:22.018 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-06-23T18:37:22.018 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-06-23T18:37:22.018 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-06-23T18:37:22.018 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi079 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir)' 2023-06-23T18:37:22.018 INFO:tasks.cephfs_test_runner: 2023-06-23T18:37:22.019 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-06-23T18:37:22.081 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=ad270baecd2e4f3485a40bd8fa4a0a47 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-06-23T18:37:22.083 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-06-23T18:37:22.092 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-06-23T18:37:22.115 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-06-23T18:37:22.115 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:37:22.116 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:37:22.145 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=435 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-06-23T18:37:22.145 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-23T18:37:22.145 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:37:22.201 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=435 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-06-23T18:37:22.202 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:37:22.202 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:37:22.223 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19111 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-06-23T18:37:22.223 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-06-23T18:37:22.224 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:37:22.282 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19111 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-06-23T18:37:22.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:22.283 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2023-06-23T18:37:22.300 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-06-23T18:37:22.300 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-06-23T18:37:22.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:22.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-06-23T18:37:22.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:22.386 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-23T18:37:22.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:22.480 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link delete ceph-brx 2023-06-23T18:37:22.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:22.583 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-06-23T18:37:22.603 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-06-23T18:37:22.604 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-06-23T18:37:22.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:22.605 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-06-23T18:37:22.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:22.685 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-06-23T18:37:22.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:37:22.780 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link delete ceph-brx 2023-06-23T18:37:22.870 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-06-23T18:37:22.879 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-06-23T18:37:22.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:37:23.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.998+0000 7f2b12f2c700 1 Processor -- start 2023-06-23T18:37:23.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.998+0000 7f2b12f2c700 1 -- start start 2023-06-23T18:37:23.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b12f2c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2b0c15a380 0x7f2b0c0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b12f2c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2b0c0cea30 0x7f2b0c0b1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b12f2c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b0c0cee20 con 0x7f2b0c15a750 2023-06-23T18:37:23.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b12f2c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b0c0b1d40 con 0x7f2b0c15a380 2023-06-23T18:37:23.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b12f2c700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b0c0b2020 con 0x7f2b0c0cea30 2023-06-23T18:37:23.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b10cc8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2b0c15a380 0x7f2b0c0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b10cc8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2b0c15a380 0x7f2b0c0ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42212/0 (socket says 172.21.15.79:42212) 2023-06-23T18:37:23.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b10cc8700 1 -- 172.21.15.79:0/4106164506 learned_addr learned my addr 172.21.15.79:0/4106164506 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:23.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b114c9700 1 --2- 172.21.15.79:0/4106164506 >> v2:172.21.15.155:3300/0 conn(0x7f2b0c0cea30 0x7f2b0c0b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b10cc8700 1 -- 172.21.15.79:0/4106164506 >> v2:172.21.15.155:3300/0 conn(0x7f2b0c0cea30 msgr2=0x7f2b0c0b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:23.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b10cc8700 1 --2- 172.21.15.79:0/4106164506 >> v2:172.21.15.155:3300/0 conn(0x7f2b0c0cea30 0x7f2b0c0b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b10cc8700 1 -- 172.21.15.79:0/4106164506 >> v1:172.21.15.155:6789/0 conn(0x7f2b0c15a750 legacy=0x7f2b0c0ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b10cc8700 1 -- 172.21.15.79:0/4106164506 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0c0ad650 con 0x7f2b0c15a380 2023-06-23T18:37:23.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b10cc8700 1 --2- 172.21.15.79:0/4106164506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c15a380 0x7f2b0c0ce4f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0c43f0 tx=0x7f2b00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6fef289dbde139 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:23.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b0b7fe700 1 -- 172.21.15.79:0/4106164506 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00005e30 con 0x7f2b0c15a380 2023-06-23T18:37:23.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b0b7fe700 1 -- 172.21.15.79:0/4106164506 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b00004030 con 0x7f2b0c15a380 2023-06-23T18:37:23.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:22.999+0000 7f2b0b7fe700 1 -- 172.21.15.79:0/4106164506 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b000043e0 con 0x7f2b0c15a380 2023-06-23T18:37:23.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.000+0000 7f2b12f2c700 1 -- 172.21.15.79:0/4106164506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c15a380 msgr2=0x7f2b0c0ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.000+0000 7f2b12f2c700 1 --2- 172.21.15.79:0/4106164506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c15a380 0x7f2b0c0ce4f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0c43f0 tx=0x7f2b00009bd0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.000+0000 7f2b12f2c700 1 -- 172.21.15.79:0/4106164506 shutdown_connections 2023-06-23T18:37:23.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.000+0000 7f2b12f2c700 1 --2- 172.21.15.79:0/4106164506 >> v2:172.21.15.155:3300/0 conn(0x7f2b0c0cea30 0x7f2b0c0b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.000+0000 7f2b12f2c700 1 --2- 172.21.15.79:0/4106164506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c15a380 0x7f2b0c0ce4f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.000+0000 7f2b12f2c700 1 -- 172.21.15.79:0/4106164506 >> 172.21.15.79:0/4106164506 conn(0x7f2b0c089fc0 msgr2=0x7f2b0c099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:23.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.000+0000 7f2b12f2c700 1 -- 172.21.15.79:0/4106164506 shutdown_connections 2023-06-23T18:37:23.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.000+0000 7f2b12f2c700 1 -- 172.21.15.79:0/4106164506 wait complete. 2023-06-23T18:37:23.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.001+0000 7f2b12f2c700 1 Processor -- start 2023-06-23T18:37:23.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.001+0000 7f2b12f2c700 1 -- start start 2023-06-23T18:37:23.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.002+0000 7f2b12f2c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c0cea30 0x7f2b0c155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.002+0000 7f2b12f2c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b0c15a380 0x7f2b0c06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.002+0000 7f2b12f2c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b0c15a750 0x7f2b0c1f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.002+0000 7f2b12f2c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c06afa0 con 0x7f2b0c0cea30 2023-06-23T18:37:23.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.002+0000 7f2b12f2c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c1f2b90 con 0x7f2b0c15a750 2023-06-23T18:37:23.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.002+0000 7f2b12f2c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c1f2e70 con 0x7f2b0c15a380 2023-06-23T18:37:23.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.002+0000 7f2b10cc8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c0cea30 0x7f2b0c155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.002+0000 7f2b10cc8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c0cea30 0x7f2b0c155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42218/0 (socket says 172.21.15.79:42218) 2023-06-23T18:37:23.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.002+0000 7f2b10cc8700 1 -- 172.21.15.79:0/2690157485 learned_addr learned my addr 172.21.15.79:0/2690157485 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:23.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.002+0000 7f2b114c9700 1 --2- 172.21.15.79:0/2690157485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b0c15a750 0x7f2b0c1f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.002+0000 7f2b0bfff700 1 --2- 172.21.15.79:0/2690157485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b0c15a380 0x7f2b0c06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.003+0000 7f2b10cc8700 1 -- 172.21.15.79:0/2690157485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b0c15a380 msgr2=0x7f2b0c06b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.003+0000 7f2b10cc8700 1 --2- 172.21.15.79:0/2690157485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b0c15a380 0x7f2b0c06b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.003+0000 7f2b10cc8700 1 -- 172.21.15.79:0/2690157485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b0c15a750 msgr2=0x7f2b0c1f2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.003+0000 7f2b10cc8700 1 --2- 172.21.15.79:0/2690157485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b0c15a750 0x7f2b0c1f2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.003+0000 7f2b10cc8700 1 -- 172.21.15.79:0/2690157485 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b00009500 con 0x7f2b0c0cea30 2023-06-23T18:37:23.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.003+0000 7f2b10cc8700 1 --2- 172.21.15.79:0/2690157485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c0cea30 0x7f2b0c155830 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2b00000c00 tx=0x7f2b00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:23.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.003+0000 7f2b097fa700 1 -- 172.21.15.79:0/2690157485 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00003c30 con 0x7f2b0c0cea30 2023-06-23T18:37:23.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.003+0000 7f2b097fa700 1 -- 172.21.15.79:0/2690157485 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b00003d90 con 0x7f2b0c0cea30 2023-06-23T18:37:23.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.003+0000 7f2b097fa700 1 -- 172.21.15.79:0/2690157485 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0001c640 con 0x7f2b0c0cea30 2023-06-23T18:37:23.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.003+0000 7f2b12f2c700 1 -- 172.21.15.79:0/2690157485 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b0c06b1b0 con 0x7f2b0c0cea30 2023-06-23T18:37:23.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.003+0000 7f2b12f2c700 1 -- 172.21.15.79:0/2690157485 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b0c155d70 con 0x7f2b0c0cea30 2023-06-23T18:37:23.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.004+0000 7f2b12f2c700 1 -- 172.21.15.79:0/2690157485 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b0c1f3090 con 0x7f2b0c0cea30 2023-06-23T18:37:23.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.004+0000 7f2b097fa700 1 -- 172.21.15.79:0/2690157485 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2b0001c7a0 con 0x7f2b0c0cea30 2023-06-23T18:37:23.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.005+0000 7f2b097fa700 1 --2- 172.21.15.79:0/2690157485 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2af006ace0 0x7f2af006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.005+0000 7f2b097fa700 1 -- 172.21.15.79:0/2690157485 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2b00089fa0 con 0x7f2b0c0cea30 2023-06-23T18:37:23.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.005+0000 7f2b0bfff700 1 --2- 172.21.15.79:0/2690157485 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2af006ace0 0x7f2af006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.005+0000 7f2b0bfff700 1 --2- 172.21.15.79:0/2690157485 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2af006ace0 0x7f2af006d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c1564c0 tx=0x7f2af8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:23.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.009+0000 7f2b097fa700 1 -- 172.21.15.79:0/2690157485 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2b000bc310 con 0x7f2b0c0cea30 2023-06-23T18:37:23.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.191+0000 7f2b12f2c700 1 -- 172.21.15.79:0/2690157485 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2b0c0ad930 con 0x7f2af006ace0 2023-06-23T18:37:23.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.194+0000 7f2b097fa700 1 -- 172.21.15.79:0/2690157485 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72873 (secure 0 0 0) 0x7f2af8001cc0 con 0x7f2af006ace0 2023-06-23T18:37:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:23.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.196+0000 7f2b12f2c700 1 -- 172.21.15.79:0/2690157485 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2af006ace0 msgr2=0x7f2af006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.196+0000 7f2b12f2c700 1 --2- 172.21.15.79:0/2690157485 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2af006ace0 0x7f2af006d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c1564c0 tx=0x7f2af8009040 comp rx=0 tx=0).stop 2023-06-23T18:37:23.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.196+0000 7f2b12f2c700 1 -- 172.21.15.79:0/2690157485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c0cea30 msgr2=0x7f2b0c155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.196+0000 7f2b12f2c700 1 --2- 172.21.15.79:0/2690157485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c0cea30 0x7f2b0c155830 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2b00000c00 tx=0x7f2b00004750 comp rx=0 tx=0).stop 2023-06-23T18:37:23.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.196+0000 7f2b12f2c700 1 -- 172.21.15.79:0/2690157485 shutdown_connections 2023-06-23T18:37:23.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.196+0000 7f2b12f2c700 1 --2- 172.21.15.79:0/2690157485 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f2af006ace0 0x7f2af006d190 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.196+0000 7f2b12f2c700 1 --2- 172.21.15.79:0/2690157485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b0c15a750 0x7f2b0c1f2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.196+0000 7f2b12f2c700 1 --2- 172.21.15.79:0/2690157485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c0cea30 0x7f2b0c155830 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.196+0000 7f2b12f2c700 1 --2- 172.21.15.79:0/2690157485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b0c15a380 0x7f2b0c06b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.196+0000 7f2b12f2c700 1 -- 172.21.15.79:0/2690157485 >> 172.21.15.79:0/2690157485 conn(0x7f2b0c089fc0 msgr2=0x7f2b0c06fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:23.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.196+0000 7f2b12f2c700 1 -- 172.21.15.79:0/2690157485 shutdown_connections 2023-06-23T18:37:23.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.196+0000 7f2b12f2c700 1 -- 172.21.15.79:0/2690157485 wait complete. 2023-06-23T18:37:23.205 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-23T18:37:23.213 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2023-06-23T18:37:22.713337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":201,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219956,"kb_used_data":4340,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754764,"statfs":{"total":773094113280,"available":772868878336,"internally_reserved":0,"allocated":4444160,"data_stored":2535846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6371,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001190"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":49,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.877629+0000","last_change":"2023-06-23T18:35:48.920009+0000","last_active":"2023-06-23T18:37:20.877629+0000","last_peered":"2023-06-23T18:37:20.877629+0000","last_clean":"2023-06-23T18:37:20.877629+0000","last_became_active":"2023-06-23T18:35:48.919686+0000","last_became_peered":"2023-06-23T18:35:48.919686+0000","last_unstale":"2023-06-23T18:37:20.877629+0000","last_undegraded":"2023-06-23T18:37:20.877629+0000","last_fullsized":"2023-06-23T18:37:20.877629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:25:08.495982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.876590+0000","last_change":"2023-06-23T18:35:47.962164+0000","last_active":"2023-06-23T18:37:20.876590+0000","last_peered":"2023-06-23T18:37:20.876590+0000","last_clean":"2023-06-23T18:37:20.876590+0000","last_became_active":"2023-06-23T18:35:47.961790+0000","last_became_peered":"2023-06-23T18:35:47.961790+0000","last_unstale":"2023-06-23T18:37:20.876590+0000","last_undegraded":"2023-06-23T18:37:20.876590+0000","last_fullsized":"2023-06-23T18:37:20.876590+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:20:49.269780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.375096+0000","last_change":"2023-06-23T18:35:47.965867+0000","last_active":"2023-06-23T18:37:20.375096+0000","last_peered":"2023-06-23T18:37:20.375096+0000","last_clean":"2023-06-23T18:37:20.375096+0000","last_became_active":"2023-06-23T18:35:47.965580+0000","last_became_peered":"2023-06-23T18:35:47.965580+0000","last_unstale":"2023-06-23T18:37:20.375096+0000","last_undegraded":"2023-06-23T18:37:20.375096+0000","last_fullsized":"2023-06-23T18:37:20.375096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:39:08.062454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":197,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:19.988264+0000","last_change":"2023-06-23T18:35:47.968534+0000","last_active":"2023-06-23T18:37:19.988264+0000","last_peered":"2023-06-23T18:37:19.988264+0000","last_clean":"2023-06-23T18:37:19.988264+0000","last_became_active":"2023-06-23T18:35:47.967508+0000","last_became_peered":"2023-06-23T18:35:47.967508+0000","last_unstale":"2023-06-23T18:37:19.988264+0000","last_undegraded":"2023-06-23T18:37:19.988264+0000","last_fullsized":"2023-06-23T18:37:19.988264+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:35:03.009522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:19.986576+0000","last_change":"2023-06-23T18:35:48.736407+0000","last_active":"2023-06-23T18:37:19.986576+0000","last_peered":"2023-06-23T18:37:19.986576+0000","last_clean":"2023-06-23T18:37:19.986576+0000","last_became_active":"2023-06-23T18:35:48.735621+0000","last_became_peered":"2023-06-23T18:35:48.735621+0000","last_unstale":"2023-06-23T18:37:19.986576+0000","last_undegraded":"2023-06-23T18:37:19.986576+0000","last_fullsized":"2023-06-23T18:37:19.986576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:47:25.406778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:19.988108+0000","last_change":"2023-06-23T18:35:48.736449+0000","last_active":"2023-06-23T18:37:19.988108+0000","last_peered":"2023-06-23T18:37:19.988108+0000","last_clean":"2023-06-23T18:37:19.988108+0000","last_became_active":"2023-06-23T18:35:48.735664+0000","last_became_peered":"2023-06-23T18:35:48.735664+0000","last_unstale":"2023-06-23T18:37:19.988108+0000","last_undegraded":"2023-06-23T18:37:19.988108+0000","last_fullsized":"2023-06-23T18:37:19.988108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:25:08.495982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":49,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:19.987247+0000","last_change":"2023-06-23T18:35:48.735713+0000","last_active":"2023-06-23T18:37:19.987247+0000","last_peered":"2023-06-23T18:37:19.987247+0000","last_clean":"2023-06-23T18:37:19.987247+0000","last_became_active":"2023-06-23T18:35:48.735236+0000","last_became_peered":"2023-06-23T18:35:48.735236+0000","last_unstale":"2023-06-23T18:37:19.987247+0000","last_undegraded":"2023-06-23T18:37:19.987247+0000","last_fullsized":"2023-06-23T18:37:19.987247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:15:44.345961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.876810+0000","last_change":"2023-06-23T18:35:48.735631+0000","last_active":"2023-06-23T18:37:20.876810+0000","last_peered":"2023-06-23T18:37:20.876810+0000","last_clean":"2023-06-23T18:37:20.876810+0000","last_became_active":"2023-06-23T18:35:48.735255+0000","last_became_peered":"2023-06-23T18:35:48.735255+0000","last_unstale":"2023-06-23T18:37:20.876810+0000","last_undegraded":"2023-06-23T18:37:20.876810+0000","last_fullsized":"2023-06-23T18:37:20.876810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:27:33.985822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.877225+0000","last_change":"2023-06-23T18:35:47.970359+0000","last_active":"2023-06-23T18:37:20.877225+0000","last_peered":"2023-06-23T18:37:20.877225+0000","last_clean":"2023-06-23T18:37:20.877225+0000","last_became_active":"2023-06-23T18:35:47.970118+0000","last_became_peered":"2023-06-23T18:35:47.970118+0000","last_unstale":"2023-06-23T18:37:20.877225+0000","last_undegraded":"2023-06-23T18:37:20.877225+0000","last_fullsized":"2023-06-23T18:37:20.877225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:14:01.922430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":1003520,"data_stored":762014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":11.071,"5min":11.071,"15min":11.071},"last":0.47899999999999998}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":11.118,"5min":11.118,"15min":11.118},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":11.103999999999999,"5min":11.103999999999999,"15min":11.103999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":0.497}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.626}]},{"osd":5,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":167233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.36799999999999999}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.48799999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.38400000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.53100000000000003}]},{"osd":5,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.432},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":995328,"data_stored":757109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.8680000000000003,"5min":7.8680000000000003,"15min":7.8680000000000003},"last":0.251}]},{"osd":2,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.6429999999999998,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":0.377},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":9.4209999999999994,"5min":9.4209999999999994,"15min":9.4209999999999994},"last":0.39100000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":5.6790000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":7.9039999999999999,"5min":7.9039999999999999,"15min":7.9039999999999999},"last":0.40500000000000003}]},{"osd":4,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":11.385999999999999,"5min":11.385999999999999,"15min":11.385999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":7.9649999999999999,"5min":7.9649999999999999,"15min":7.9649999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":11.259,"5min":11.259,"15min":11.259},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":7.9909999999999997,"5min":7.9909999999999997,"15min":7.9909999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":11.346,"5min":11.346,"15min":11.346},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":7.9359999999999999,"5min":7.9359999999999999,"15min":7.9359999999999999},"last":0.73799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":165324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":16.004999999999999,"5min":16.004999999999999,"15min":16.004999999999999},"last":0.24099999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":16.073,"5min":16.073,"15min":16.073},"last":0.22700000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":8.2460000000000004,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":8.2129999999999992,"5min":8.2129999999999992,"15min":8.2129999999999992},"last":0.32300000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":16.036999999999999,"5min":16.036999999999999,"15min":16.036999999999999},"last":0.40000000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":8.0600000000000005,"5min":8.0600000000000005,"15min":8.0600000000000005},"last":0.443},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":8.2739999999999991,"5min":8.2739999999999991,"15min":8.2739999999999991},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":8.3320000000000007,"5min":8.3320000000000007,"15min":8.3320000000000007},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":8.0299999999999994,"5min":8.0299999999999994,"15min":8.0299999999999994},"last":0.47199999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.3030000000000008,"5min":8.3030000000000008,"15min":8.3030000000000008},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":8.3800000000000008,"5min":8.3800000000000008,"15min":8.3800000000000008},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":8.3580000000000005,"5min":8.3580000000000005,"15min":8.3580000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":8.0839999999999996,"5min":8.0839999999999996,"15min":8.0839999999999996},"last":0.55700000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":409600,"data_stored":171646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.246},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.27800000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":5.0899999999999999,"5min":5.0899999999999999,"15min":5.0899999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.316}]},{"osd":4,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.501}]},{"osd":6,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.51000000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":172258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":4.359,"5min":4.359,"15min":4.359},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.78300000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.82499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":167255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.57999999999999996}]},{"osd":1,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.498}]},{"osd":2,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.439}]},{"osd":3,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.375},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48799999999999999}]},{"osd":7,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60099999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":173007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40300000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.57599999999999996}]},{"osd":6,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.61199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:37:23.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:37:23.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.333+0000 7fd3aa7dd700 1 Processor -- start 2023-06-23T18:37:23.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.333+0000 7fd3aa7dd700 1 -- start start 2023-06-23T18:37:23.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.333+0000 7fd3aa7dd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd3a415a380 0x7fd3a40ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.333+0000 7fd3aa7dd700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a40cee20 con 0x7fd3a40cea30 2023-06-23T18:37:23.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.333+0000 7fd3aa7dd700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a40b1d40 con 0x7fd3a415a750 2023-06-23T18:37:23.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.333+0000 7fd3aa7dd700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a40b2020 con 0x7fd3a415a380 2023-06-23T18:37:23.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.334+0000 7fd3a3fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd3a415a380 0x7fd3a40ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.334+0000 7fd3a8d7a700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd3a40cea30 0x7fd3a40b1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45272/0 (socket says 172.21.15.79:45272) 2023-06-23T18:37:23.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.334+0000 7fd3a8d7a700 1 -- 172.21.15.79:0/1448153078 learned_addr learned my addr 172.21.15.79:0/1448153078 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:23.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.334+0000 7fd3a3fff700 1 -- 172.21.15.79:0/1448153078 >> v1:172.21.15.155:6789/0 conn(0x7fd3a415a750 legacy=0x7fd3a40ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.334+0000 7fd3a3fff700 1 -- 172.21.15.79:0/1448153078 >> v1:172.21.15.79:6789/0 conn(0x7fd3a40cea30 legacy=0x7fd3a40b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.334+0000 7fd3a3fff700 1 -- 172.21.15.79:0/1448153078 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a40ad650 con 0x7fd3a415a380 2023-06-23T18:37:23.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.334+0000 7fd3a3fff700 1 --2- 172.21.15.79:0/1448153078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a415a380 0x7fd3a40ce4f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd394009820 tx=0x7fd394009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6183e716af8d8df server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:23.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.334+0000 7fd3a2ffd700 1 -- 172.21.15.79:0/1448153078 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd394005e30 con 0x7fd3a415a380 2023-06-23T18:37:23.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.334+0000 7fd3a2ffd700 1 -- 172.21.15.79:0/1448153078 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd394006030 con 0x7fd3a415a380 2023-06-23T18:37:23.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.334+0000 7fd3a2ffd700 1 -- 172.21.15.79:0/1448153078 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3940063e0 con 0x7fd3a415a380 2023-06-23T18:37:23.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.335+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1448153078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a415a380 msgr2=0x7fd3a40ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.335+0000 7fd3aa7dd700 1 --2- 172.21.15.79:0/1448153078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a415a380 0x7fd3a40ce4f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd394009820 tx=0x7fd394009b30 comp rx=0 tx=0).stop 2023-06-23T18:37:23.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.335+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1448153078 shutdown_connections 2023-06-23T18:37:23.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.335+0000 7fd3aa7dd700 1 --2- 172.21.15.79:0/1448153078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a415a380 0x7fd3a40ce4f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.335+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1448153078 >> 172.21.15.79:0/1448153078 conn(0x7fd3a4089fc0 msgr2=0x7fd3a4099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:23.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.335+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1448153078 shutdown_connections 2023-06-23T18:37:23.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.335+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1448153078 wait complete. 2023-06-23T18:37:23.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.336+0000 7fd3aa7dd700 1 Processor -- start 2023-06-23T18:37:23.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.336+0000 7fd3aa7dd700 1 -- start start 2023-06-23T18:37:23.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3aa7dd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a40cea30 0x7fd3a406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3aa7dd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a415a380 0x7fd3a415cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3aa7dd700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a415a750 0x7fd3a415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3aa7dd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a40cc260 con 0x7fd3a415a380 2023-06-23T18:37:23.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3aa7dd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a415d9b0 con 0x7fd3a40cea30 2023-06-23T18:37:23.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3aa7dd700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a40cbf30 con 0x7fd3a415a750 2023-06-23T18:37:23.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3a37fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a415a380 0x7fd3a415cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3a3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a40cea30 0x7fd3a406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3a37fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a415a380 0x7fd3a415cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42234/0 (socket says 172.21.15.79:42234) 2023-06-23T18:37:23.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3a37fe700 1 -- 172.21.15.79:0/1242766926 learned_addr learned my addr 172.21.15.79:0/1242766926 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:23.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3a8d7a700 1 --2- 172.21.15.79:0/1242766926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a415a750 0x7fd3a415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3a37fe700 1 -- 172.21.15.79:0/1242766926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a415a750 msgr2=0x7fd3a415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3a37fe700 1 --2- 172.21.15.79:0/1242766926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a415a750 0x7fd3a415b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3a37fe700 1 -- 172.21.15.79:0/1242766926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a40cea30 msgr2=0x7fd3a406cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3a37fe700 1 --2- 172.21.15.79:0/1242766926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a40cea30 0x7fd3a406cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3a37fe700 1 -- 172.21.15.79:0/1242766926 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd394009430 con 0x7fd3a415a380 2023-06-23T18:37:23.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.337+0000 7fd3a37fe700 1 --2- 172.21.15.79:0/1242766926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a415a380 0x7fd3a415cc10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd39800c400 tx=0x7fd39800c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:23.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.338+0000 7fd3a0ff9700 1 -- 172.21.15.79:0/1242766926 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3980052d0 con 0x7fd3a415a380 2023-06-23T18:37:23.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.338+0000 7fd3a0ff9700 1 -- 172.21.15.79:0/1242766926 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd398005430 con 0x7fd3a415a380 2023-06-23T18:37:23.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.338+0000 7fd3a0ff9700 1 -- 172.21.15.79:0/1242766926 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3980038c0 con 0x7fd3a415a380 2023-06-23T18:37:23.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.338+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1242766926 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a415d150 con 0x7fd3a415a380 2023-06-23T18:37:23.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.338+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1242766926 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a415d6d0 con 0x7fd3a415a380 2023-06-23T18:37:23.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.338+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1242766926 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a40cc3d0 con 0x7fd3a415a380 2023-06-23T18:37:23.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.339+0000 7fd3a0ff9700 1 -- 172.21.15.79:0/1242766926 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd398003a20 con 0x7fd3a415a380 2023-06-23T18:37:23.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.339+0000 7fd3a0ff9700 1 --2- 172.21.15.79:0/1242766926 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd39006ace0 0x7fd39006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.339+0000 7fd3a0ff9700 1 -- 172.21.15.79:0/1242766926 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd398088370 con 0x7fd3a415a380 2023-06-23T18:37:23.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.339+0000 7fd3a3fff700 1 --2- 172.21.15.79:0/1242766926 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd39006ace0 0x7fd39006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.340+0000 7fd3a3fff700 1 --2- 172.21.15.79:0/1242766926 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd39006ace0 0x7fd39006d190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd394013040 tx=0x7fd394008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:23.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.344+0000 7fd3a0ff9700 1 -- 172.21.15.79:0/1242766926 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd3980537b0 con 0x7fd3a415a380 2023-06-23T18:37:23.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.525+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1242766926 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd3a40ad930 con 0x7fd39006ace0 2023-06-23T18:37:23.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.528+0000 7fd3a0ff9700 1 -- 172.21.15.79:0/1242766926 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72873 (secure 0 0 0) 0x7fd394006250 con 0x7fd39006ace0 2023-06-23T18:37:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:23.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.530+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1242766926 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd39006ace0 msgr2=0x7fd39006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.530+0000 7fd3aa7dd700 1 --2- 172.21.15.79:0/1242766926 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd39006ace0 0x7fd39006d190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd394013040 tx=0x7fd394008070 comp rx=0 tx=0).stop 2023-06-23T18:37:23.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.530+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1242766926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a415a380 msgr2=0x7fd3a415cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.530+0000 7fd3aa7dd700 1 --2- 172.21.15.79:0/1242766926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a415a380 0x7fd3a415cc10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd39800c400 tx=0x7fd39800c710 comp rx=0 tx=0).stop 2023-06-23T18:37:23.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.530+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1242766926 shutdown_connections 2023-06-23T18:37:23.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.530+0000 7fd3aa7dd700 1 --2- 172.21.15.79:0/1242766926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a415a750 0x7fd3a415b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.530+0000 7fd3aa7dd700 1 --2- 172.21.15.79:0/1242766926 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fd39006ace0 0x7fd39006d190 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.530+0000 7fd3aa7dd700 1 --2- 172.21.15.79:0/1242766926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a40cea30 0x7fd3a406cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.530+0000 7fd3aa7dd700 1 --2- 172.21.15.79:0/1242766926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a415a380 0x7fd3a415cc10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.530+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1242766926 >> 172.21.15.79:0/1242766926 conn(0x7fd3a4089fc0 msgr2=0x7fd3a40afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:23.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.530+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1242766926 shutdown_connections 2023-06-23T18:37:23.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.530+0000 7fd3aa7dd700 1 -- 172.21.15.79:0/1242766926 wait complete. 2023-06-23T18:37:23.538 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-23T18:37:23.547 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2023-06-23T18:37:22.713337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":201,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219956,"kb_used_data":4340,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754764,"statfs":{"total":773094113280,"available":772868878336,"internally_reserved":0,"allocated":4444160,"data_stored":2535846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6371,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001190"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":49,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.877629+0000","last_change":"2023-06-23T18:35:48.920009+0000","last_active":"2023-06-23T18:37:20.877629+0000","last_peered":"2023-06-23T18:37:20.877629+0000","last_clean":"2023-06-23T18:37:20.877629+0000","last_became_active":"2023-06-23T18:35:48.919686+0000","last_became_peered":"2023-06-23T18:35:48.919686+0000","last_unstale":"2023-06-23T18:37:20.877629+0000","last_undegraded":"2023-06-23T18:37:20.877629+0000","last_fullsized":"2023-06-23T18:37:20.877629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:25:08.495982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.876590+0000","last_change":"2023-06-23T18:35:47.962164+0000","last_active":"2023-06-23T18:37:20.876590+0000","last_peered":"2023-06-23T18:37:20.876590+0000","last_clean":"2023-06-23T18:37:20.876590+0000","last_became_active":"2023-06-23T18:35:47.961790+0000","last_became_peered":"2023-06-23T18:35:47.961790+0000","last_unstale":"2023-06-23T18:37:20.876590+0000","last_undegraded":"2023-06-23T18:37:20.876590+0000","last_fullsized":"2023-06-23T18:37:20.876590+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:20:49.269780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.375096+0000","last_change":"2023-06-23T18:35:47.965867+0000","last_active":"2023-06-23T18:37:20.375096+0000","last_peered":"2023-06-23T18:37:20.375096+0000","last_clean":"2023-06-23T18:37:20.375096+0000","last_became_active":"2023-06-23T18:35:47.965580+0000","last_became_peered":"2023-06-23T18:35:47.965580+0000","last_unstale":"2023-06-23T18:37:20.375096+0000","last_undegraded":"2023-06-23T18:37:20.375096+0000","last_fullsized":"2023-06-23T18:37:20.375096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:39:08.062454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":197,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:19.988264+0000","last_change":"2023-06-23T18:35:47.968534+0000","last_active":"2023-06-23T18:37:19.988264+0000","last_peered":"2023-06-23T18:37:19.988264+0000","last_clean":"2023-06-23T18:37:19.988264+0000","last_became_active":"2023-06-23T18:35:47.967508+0000","last_became_peered":"2023-06-23T18:35:47.967508+0000","last_unstale":"2023-06-23T18:37:19.988264+0000","last_undegraded":"2023-06-23T18:37:19.988264+0000","last_fullsized":"2023-06-23T18:37:19.988264+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:35:03.009522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:19.986576+0000","last_change":"2023-06-23T18:35:48.736407+0000","last_active":"2023-06-23T18:37:19.986576+0000","last_peered":"2023-06-23T18:37:19.986576+0000","last_clean":"2023-06-23T18:37:19.986576+0000","last_became_active":"2023-06-23T18:35:48.735621+0000","last_became_peered":"2023-06-23T18:35:48.735621+0000","last_unstale":"2023-06-23T18:37:19.986576+0000","last_undegraded":"2023-06-23T18:37:19.986576+0000","last_fullsized":"2023-06-23T18:37:19.986576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:47:25.406778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:19.988108+0000","last_change":"2023-06-23T18:35:48.736449+0000","last_active":"2023-06-23T18:37:19.988108+0000","last_peered":"2023-06-23T18:37:19.988108+0000","last_clean":"2023-06-23T18:37:19.988108+0000","last_became_active":"2023-06-23T18:35:48.735664+0000","last_became_peered":"2023-06-23T18:35:48.735664+0000","last_unstale":"2023-06-23T18:37:19.988108+0000","last_undegraded":"2023-06-23T18:37:19.988108+0000","last_fullsized":"2023-06-23T18:37:19.988108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:25:08.495982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":49,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:19.987247+0000","last_change":"2023-06-23T18:35:48.735713+0000","last_active":"2023-06-23T18:37:19.987247+0000","last_peered":"2023-06-23T18:37:19.987247+0000","last_clean":"2023-06-23T18:37:19.987247+0000","last_became_active":"2023-06-23T18:35:48.735236+0000","last_became_peered":"2023-06-23T18:35:48.735236+0000","last_unstale":"2023-06-23T18:37:19.987247+0000","last_undegraded":"2023-06-23T18:37:19.987247+0000","last_fullsized":"2023-06-23T18:37:19.987247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:15:44.345961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.876810+0000","last_change":"2023-06-23T18:35:48.735631+0000","last_active":"2023-06-23T18:37:20.876810+0000","last_peered":"2023-06-23T18:37:20.876810+0000","last_clean":"2023-06-23T18:37:20.876810+0000","last_became_active":"2023-06-23T18:35:48.735255+0000","last_became_peered":"2023-06-23T18:35:48.735255+0000","last_unstale":"2023-06-23T18:37:20.876810+0000","last_undegraded":"2023-06-23T18:37:20.876810+0000","last_fullsized":"2023-06-23T18:37:20.876810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:27:33.985822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.877225+0000","last_change":"2023-06-23T18:35:47.970359+0000","last_active":"2023-06-23T18:37:20.877225+0000","last_peered":"2023-06-23T18:37:20.877225+0000","last_clean":"2023-06-23T18:37:20.877225+0000","last_became_active":"2023-06-23T18:35:47.970118+0000","last_became_peered":"2023-06-23T18:35:47.970118+0000","last_unstale":"2023-06-23T18:37:20.877225+0000","last_undegraded":"2023-06-23T18:37:20.877225+0000","last_fullsized":"2023-06-23T18:37:20.877225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:14:01.922430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":1003520,"data_stored":762014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":11.071,"5min":11.071,"15min":11.071},"last":0.47899999999999998}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":11.118,"5min":11.118,"15min":11.118},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":11.103999999999999,"5min":11.103999999999999,"15min":11.103999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":0.497}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.626}]},{"osd":5,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":167233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.36799999999999999}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.48799999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.38400000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.53100000000000003}]},{"osd":5,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.432},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":995328,"data_stored":757109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.8680000000000003,"5min":7.8680000000000003,"15min":7.8680000000000003},"last":0.251}]},{"osd":2,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.6429999999999998,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":0.377},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":9.4209999999999994,"5min":9.4209999999999994,"15min":9.4209999999999994},"last":0.39100000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":5.6790000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":7.9039999999999999,"5min":7.9039999999999999,"15min":7.9039999999999999},"last":0.40500000000000003}]},{"osd":4,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":11.385999999999999,"5min":11.385999999999999,"15min":11.385999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":7.9649999999999999,"5min":7.9649999999999999,"15min":7.9649999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":11.259,"5min":11.259,"15min":11.259},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":7.9909999999999997,"5min":7.9909999999999997,"15min":7.9909999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":11.346,"5min":11.346,"15min":11.346},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":7.9359999999999999,"5min":7.9359999999999999,"15min":7.9359999999999999},"last":0.73799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":165324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":16.004999999999999,"5min":16.004999999999999,"15min":16.004999999999999},"last":0.24099999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":16.073,"5min":16.073,"15min":16.073},"last":0.22700000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":8.2460000000000004,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":8.2129999999999992,"5min":8.2129999999999992,"15min":8.2129999999999992},"last":0.32300000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":16.036999999999999,"5min":16.036999999999999,"15min":16.036999999999999},"last":0.40000000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":8.0600000000000005,"5min":8.0600000000000005,"15min":8.0600000000000005},"last":0.443},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":8.2739999999999991,"5min":8.2739999999999991,"15min":8.2739999999999991},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":8.3320000000000007,"5min":8.3320000000000007,"15min":8.3320000000000007},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":8.0299999999999994,"5min":8.0299999999999994,"15min":8.0299999999999994},"last":0.47199999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.3030000000000008,"5min":8.3030000000000008,"15min":8.3030000000000008},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":8.3800000000000008,"5min":8.3800000000000008,"15min":8.3800000000000008},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":8.3580000000000005,"5min":8.3580000000000005,"15min":8.3580000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":8.0839999999999996,"5min":8.0839999999999996,"15min":8.0839999999999996},"last":0.55700000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":409600,"data_stored":171646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.246},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.27800000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":5.0899999999999999,"5min":5.0899999999999999,"15min":5.0899999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.316}]},{"osd":4,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.501}]},{"osd":6,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.51000000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":172258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":4.359,"5min":4.359,"15min":4.359},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.78300000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.82499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":167255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.57999999999999996}]},{"osd":1,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.498}]},{"osd":2,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.439}]},{"osd":3,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.375},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48799999999999999}]},{"osd":7,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60099999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":173007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40300000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.57599999999999996}]},{"osd":6,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.61199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:37:23.549 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-06-23T18:37:23.550 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:37:23.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.664+0000 7f1fd3d07700 1 Processor -- start 2023-06-23T18:37:23.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.665+0000 7f1fd3d07700 1 -- start start 2023-06-23T18:37:23.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.665+0000 7f1fd3d07700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1fcc0b1270 0x7f1fcc15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.665+0000 7f1fd3d07700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fcc0aee30 0x7f1fcc15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.666+0000 7f1fd3d07700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc1608f0 con 0x7f1fcc071490 2023-06-23T18:37:23.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.666+0000 7f1fd3d07700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc160e20 con 0x7f1fcc0b1270 2023-06-23T18:37:23.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.666+0000 7f1fd3d07700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0cd4f0 con 0x7f1fcc0aee30 2023-06-23T18:37:23.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.666+0000 7f1fd22a4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fcc0aee30 0x7f1fcc15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.666+0000 7f1fd12a2700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1fcc0b1270 0x7f1fcc15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.666+0000 7f1fd22a4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fcc0aee30 0x7f1fcc15cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:35014/0 (socket says 172.21.15.79:35014) 2023-06-23T18:37:23.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.666+0000 7f1fd12a2700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1fcc0b1270 0x7f1fcc15c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:48468/0 (socket says 172.21.15.79:48468) 2023-06-23T18:37:23.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.666+0000 7f1fd22a4700 1 -- 172.21.15.79:0/3614684674 learned_addr learned my addr 172.21.15.79:0/3614684674 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:23.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.666+0000 7f1fd1aa3700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f1fcc071490 0x7f1fcc071860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:53550/0 (socket says 172.21.15.79:53550) 2023-06-23T18:37:23.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.666+0000 7f1fd22a4700 1 -- 172.21.15.79:0/3614684674 >> v2:172.21.15.155:3300/0 conn(0x7f1fcc0b1270 msgr2=0x7f1fcc15c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:23.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.666+0000 7f1fd22a4700 1 --2- 172.21.15.79:0/3614684674 >> v2:172.21.15.155:3300/0 conn(0x7f1fcc0b1270 0x7f1fcc15c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.666+0000 7f1fd22a4700 1 -- 172.21.15.79:0/3614684674 >> v1:172.21.15.155:6789/0 conn(0x7f1fcc071490 legacy=0x7f1fcc071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.666+0000 7f1fd22a4700 1 -- 172.21.15.79:0/3614684674 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc160b40 con 0x7f1fcc0aee30 2023-06-23T18:37:23.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.666+0000 7f1fd22a4700 1 --2- 172.21.15.79:0/3614684674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fcc0aee30 0x7f1fcc15cf30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009e50 tx=0x7f1fc400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21d1387c56626dd4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:23.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.667+0000 7f1fd0aa1700 1 -- 172.21.15.79:0/3614684674 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc400e070 con 0x7f1fcc0aee30 2023-06-23T18:37:23.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.667+0000 7f1fd0aa1700 1 -- 172.21.15.79:0/3614684674 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc400b910 con 0x7f1fcc0aee30 2023-06-23T18:37:23.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.667+0000 7f1fd0aa1700 1 -- 172.21.15.79:0/3614684674 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc400bcc0 con 0x7f1fcc0aee30 2023-06-23T18:37:23.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.667+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3614684674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fcc0aee30 msgr2=0x7f1fcc15cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.667+0000 7f1fd3d07700 1 --2- 172.21.15.79:0/3614684674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fcc0aee30 0x7f1fcc15cf30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009e50 tx=0x7f1fc400b330 comp rx=0 tx=0).stop 2023-06-23T18:37:23.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.667+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3614684674 shutdown_connections 2023-06-23T18:37:23.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.667+0000 7f1fd3d07700 1 --2- 172.21.15.79:0/3614684674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fcc0aee30 0x7f1fcc15cf30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.667+0000 7f1fd3d07700 1 --2- 172.21.15.79:0/3614684674 >> v2:172.21.15.155:3300/0 conn(0x7f1fcc0b1270 0x7f1fcc15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.667+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3614684674 >> 172.21.15.79:0/3614684674 conn(0x7f1fcc06d660 msgr2=0x7f1fcc06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:23.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.667+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3614684674 shutdown_connections 2023-06-23T18:37:23.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.667+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3614684674 wait complete. 2023-06-23T18:37:23.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.668+0000 7f1fd3d07700 1 Processor -- start 2023-06-23T18:37:23.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.668+0000 7f1fd3d07700 1 -- start start 2023-06-23T18:37:23.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.669+0000 7f1fd3d07700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1fcc071490 0x7f1fcc0c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.669+0000 7f1fd3d07700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fcc0aee30 0x7f1fcc150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.669+0000 7f1fd3d07700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc0b1270 0x7f1fcc1570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.669+0000 7f1fd3d07700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0c5e00 con 0x7f1fcc071490 2023-06-23T18:37:23.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.669+0000 7f1fd3d07700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0c45d0 con 0x7f1fcc0b1270 2023-06-23T18:37:23.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.669+0000 7f1fd3d07700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc158190 con 0x7f1fcc0aee30 2023-06-23T18:37:23.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.669+0000 7f1fd1aa3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1fcc071490 0x7f1fcc0c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.669+0000 7f1fd1aa3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1fcc071490 0x7f1fcc0c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42246/0 (socket says 172.21.15.79:42246) 2023-06-23T18:37:23.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.669+0000 7f1fd1aa3700 1 -- 172.21.15.79:0/3144780808 learned_addr learned my addr 172.21.15.79:0/3144780808 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:23.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.669+0000 7f1fd12a2700 1 --2- 172.21.15.79:0/3144780808 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fcc0aee30 0x7f1fcc150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.669+0000 7f1fd22a4700 1 --2- 172.21.15.79:0/3144780808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc0b1270 0x7f1fcc1570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.670+0000 7f1fd1aa3700 1 -- 172.21.15.79:0/3144780808 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fcc0aee30 msgr2=0x7f1fcc150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.670+0000 7f1fd1aa3700 1 --2- 172.21.15.79:0/3144780808 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fcc0aee30 0x7f1fcc150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.670+0000 7f1fd1aa3700 1 -- 172.21.15.79:0/3144780808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc0b1270 msgr2=0x7f1fcc1570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.670+0000 7f1fd1aa3700 1 --2- 172.21.15.79:0/3144780808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc0b1270 0x7f1fcc1570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.670+0000 7f1fd1aa3700 1 -- 172.21.15.79:0/3144780808 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc4009b30 con 0x7f1fcc071490 2023-06-23T18:37:23.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.670+0000 7f1fd1aa3700 1 --2- 172.21.15.79:0/3144780808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1fcc071490 0x7f1fcc0c58c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c500 tx=0x7f1fbc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:23.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.670+0000 7f1fc27fc700 1 -- 172.21.15.79:0/3144780808 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc005cb0 con 0x7f1fcc071490 2023-06-23T18:37:23.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.670+0000 7f1fc27fc700 1 -- 172.21.15.79:0/3144780808 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc005e10 con 0x7f1fcc071490 2023-06-23T18:37:23.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.670+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3144780808 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc0c4820 con 0x7f1fcc071490 2023-06-23T18:37:23.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.670+0000 7f1fc27fc700 1 -- 172.21.15.79:0/3144780808 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0072d0 con 0x7f1fcc071490 2023-06-23T18:37:23.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.670+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3144780808 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc0c4de0 con 0x7f1fcc071490 2023-06-23T18:37:23.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.671+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3144780808 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc0724a0 con 0x7f1fcc071490 2023-06-23T18:37:23.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.671+0000 7f1fc27fc700 1 -- 172.21.15.79:0/3144780808 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1fbc007430 con 0x7f1fcc071490 2023-06-23T18:37:23.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.672+0000 7f1fc27fc700 1 --2- 172.21.15.79:0/3144780808 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1fb006ace0 0x7f1fb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:23.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.672+0000 7f1fc27fc700 1 -- 172.21.15.79:0/3144780808 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1fbc088340 con 0x7f1fcc071490 2023-06-23T18:37:23.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.672+0000 7f1fd12a2700 1 --2- 172.21.15.79:0/3144780808 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1fb006ace0 0x7f1fb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:23.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.672+0000 7f1fd12a2700 1 --2- 172.21.15.79:0/3144780808 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1fb006ace0 0x7f1fb006d190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc1f5760 tx=0x7f1fc8009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:23.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.676+0000 7f1fc27fc700 1 -- 172.21.15.79:0/3144780808 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1fbc0537a0 con 0x7f1fcc071490 2023-06-23T18:37:23.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.859+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3144780808 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1fcc0cd790 con 0x7f1fb006ace0 2023-06-23T18:37:23.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.862+0000 7f1fc27fc700 1 -- 172.21.15.79:0/3144780808 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72873 (secure 0 0 0) 0x7f1fc8011760 con 0x7f1fb006ace0 2023-06-23T18:37:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:23.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.864+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3144780808 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1fb006ace0 msgr2=0x7f1fb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.864+0000 7f1fd3d07700 1 --2- 172.21.15.79:0/3144780808 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1fb006ace0 0x7f1fb006d190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc1f5760 tx=0x7f1fc8009210 comp rx=0 tx=0).stop 2023-06-23T18:37:23.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.864+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3144780808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1fcc071490 msgr2=0x7f1fcc0c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:23.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.864+0000 7f1fd3d07700 1 --2- 172.21.15.79:0/3144780808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1fcc071490 0x7f1fcc0c58c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c500 tx=0x7f1fbc00c810 comp rx=0 tx=0).stop 2023-06-23T18:37:23.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.864+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3144780808 shutdown_connections 2023-06-23T18:37:23.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.864+0000 7f1fd3d07700 1 --2- 172.21.15.79:0/3144780808 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f1fb006ace0 0x7f1fb006d190 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.864+0000 7f1fd3d07700 1 --2- 172.21.15.79:0/3144780808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc0b1270 0x7f1fcc1570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.864+0000 7f1fd3d07700 1 --2- 172.21.15.79:0/3144780808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1fcc071490 0x7f1fcc0c58c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.864+0000 7f1fd3d07700 1 --2- 172.21.15.79:0/3144780808 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fcc0aee30 0x7f1fcc150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:23.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.864+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3144780808 >> 172.21.15.79:0/3144780808 conn(0x7f1fcc06d660 msgr2=0x7f1fcc151e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:23.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.864+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3144780808 shutdown_connections 2023-06-23T18:37:23.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.864+0000 7f1fd3d07700 1 -- 172.21.15.79:0/3144780808 wait complete. 2023-06-23T18:37:23.872 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-23T18:37:23.880 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2023-06-23T18:37:22.713337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":201,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219956,"kb_used_data":4340,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754764,"statfs":{"total":773094113280,"available":772868878336,"internally_reserved":0,"allocated":4444160,"data_stored":2535846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6371,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001190"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":49,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.877629+0000","last_change":"2023-06-23T18:35:48.920009+0000","last_active":"2023-06-23T18:37:20.877629+0000","last_peered":"2023-06-23T18:37:20.877629+0000","last_clean":"2023-06-23T18:37:20.877629+0000","last_became_active":"2023-06-23T18:35:48.919686+0000","last_became_peered":"2023-06-23T18:35:48.919686+0000","last_unstale":"2023-06-23T18:37:20.877629+0000","last_undegraded":"2023-06-23T18:37:20.877629+0000","last_fullsized":"2023-06-23T18:37:20.877629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:25:08.495982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.876590+0000","last_change":"2023-06-23T18:35:47.962164+0000","last_active":"2023-06-23T18:37:20.876590+0000","last_peered":"2023-06-23T18:37:20.876590+0000","last_clean":"2023-06-23T18:37:20.876590+0000","last_became_active":"2023-06-23T18:35:47.961790+0000","last_became_peered":"2023-06-23T18:35:47.961790+0000","last_unstale":"2023-06-23T18:37:20.876590+0000","last_undegraded":"2023-06-23T18:37:20.876590+0000","last_fullsized":"2023-06-23T18:37:20.876590+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:20:49.269780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.375096+0000","last_change":"2023-06-23T18:35:47.965867+0000","last_active":"2023-06-23T18:37:20.375096+0000","last_peered":"2023-06-23T18:37:20.375096+0000","last_clean":"2023-06-23T18:37:20.375096+0000","last_became_active":"2023-06-23T18:35:47.965580+0000","last_became_peered":"2023-06-23T18:35:47.965580+0000","last_unstale":"2023-06-23T18:37:20.375096+0000","last_undegraded":"2023-06-23T18:37:20.375096+0000","last_fullsized":"2023-06-23T18:37:20.375096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:39:08.062454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":197,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:19.988264+0000","last_change":"2023-06-23T18:35:47.968534+0000","last_active":"2023-06-23T18:37:19.988264+0000","last_peered":"2023-06-23T18:37:19.988264+0000","last_clean":"2023-06-23T18:37:19.988264+0000","last_became_active":"2023-06-23T18:35:47.967508+0000","last_became_peered":"2023-06-23T18:35:47.967508+0000","last_unstale":"2023-06-23T18:37:19.988264+0000","last_undegraded":"2023-06-23T18:37:19.988264+0000","last_fullsized":"2023-06-23T18:37:19.988264+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:35:03.009522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:19.986576+0000","last_change":"2023-06-23T18:35:48.736407+0000","last_active":"2023-06-23T18:37:19.986576+0000","last_peered":"2023-06-23T18:37:19.986576+0000","last_clean":"2023-06-23T18:37:19.986576+0000","last_became_active":"2023-06-23T18:35:48.735621+0000","last_became_peered":"2023-06-23T18:35:48.735621+0000","last_unstale":"2023-06-23T18:37:19.986576+0000","last_undegraded":"2023-06-23T18:37:19.986576+0000","last_fullsized":"2023-06-23T18:37:19.986576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:47:25.406778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:19.988108+0000","last_change":"2023-06-23T18:35:48.736449+0000","last_active":"2023-06-23T18:37:19.988108+0000","last_peered":"2023-06-23T18:37:19.988108+0000","last_clean":"2023-06-23T18:37:19.988108+0000","last_became_active":"2023-06-23T18:35:48.735664+0000","last_became_peered":"2023-06-23T18:35:48.735664+0000","last_unstale":"2023-06-23T18:37:19.988108+0000","last_undegraded":"2023-06-23T18:37:19.988108+0000","last_fullsized":"2023-06-23T18:37:19.988108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:25:08.495982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":49,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:19.987247+0000","last_change":"2023-06-23T18:35:48.735713+0000","last_active":"2023-06-23T18:37:19.987247+0000","last_peered":"2023-06-23T18:37:19.987247+0000","last_clean":"2023-06-23T18:37:19.987247+0000","last_became_active":"2023-06-23T18:35:48.735236+0000","last_became_peered":"2023-06-23T18:35:48.735236+0000","last_unstale":"2023-06-23T18:37:19.987247+0000","last_undegraded":"2023-06-23T18:37:19.987247+0000","last_fullsized":"2023-06-23T18:37:19.987247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:15:44.345961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.876810+0000","last_change":"2023-06-23T18:35:48.735631+0000","last_active":"2023-06-23T18:37:20.876810+0000","last_peered":"2023-06-23T18:37:20.876810+0000","last_clean":"2023-06-23T18:37:20.876810+0000","last_became_active":"2023-06-23T18:35:48.735255+0000","last_became_peered":"2023-06-23T18:35:48.735255+0000","last_unstale":"2023-06-23T18:37:20.876810+0000","last_undegraded":"2023-06-23T18:37:20.876810+0000","last_fullsized":"2023-06-23T18:37:20.876810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:27:33.985822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.877225+0000","last_change":"2023-06-23T18:35:47.970359+0000","last_active":"2023-06-23T18:37:20.877225+0000","last_peered":"2023-06-23T18:37:20.877225+0000","last_clean":"2023-06-23T18:37:20.877225+0000","last_became_active":"2023-06-23T18:35:47.970118+0000","last_became_peered":"2023-06-23T18:35:47.970118+0000","last_unstale":"2023-06-23T18:37:20.877225+0000","last_undegraded":"2023-06-23T18:37:20.877225+0000","last_fullsized":"2023-06-23T18:37:20.877225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:14:01.922430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":1003520,"data_stored":762014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":11.071,"5min":11.071,"15min":11.071},"last":0.47899999999999998}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":11.118,"5min":11.118,"15min":11.118},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":11.103999999999999,"5min":11.103999999999999,"15min":11.103999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":0.497}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.626}]},{"osd":5,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":167233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.36799999999999999}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.48799999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.38400000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.53100000000000003}]},{"osd":5,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.432},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":995328,"data_stored":757109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.8680000000000003,"5min":7.8680000000000003,"15min":7.8680000000000003},"last":0.251}]},{"osd":2,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.6429999999999998,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":0.377},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":9.4209999999999994,"5min":9.4209999999999994,"15min":9.4209999999999994},"last":0.39100000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":5.6790000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":7.9039999999999999,"5min":7.9039999999999999,"15min":7.9039999999999999},"last":0.40500000000000003}]},{"osd":4,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":11.385999999999999,"5min":11.385999999999999,"15min":11.385999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":7.9649999999999999,"5min":7.9649999999999999,"15min":7.9649999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":11.259,"5min":11.259,"15min":11.259},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":7.9909999999999997,"5min":7.9909999999999997,"15min":7.9909999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":11.346,"5min":11.346,"15min":11.346},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":7.9359999999999999,"5min":7.9359999999999999,"15min":7.9359999999999999},"last":0.73799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":165324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":16.004999999999999,"5min":16.004999999999999,"15min":16.004999999999999},"last":0.24099999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":16.073,"5min":16.073,"15min":16.073},"last":0.22700000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":8.2460000000000004,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":8.2129999999999992,"5min":8.2129999999999992,"15min":8.2129999999999992},"last":0.32300000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":16.036999999999999,"5min":16.036999999999999,"15min":16.036999999999999},"last":0.40000000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":8.0600000000000005,"5min":8.0600000000000005,"15min":8.0600000000000005},"last":0.443},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":8.2739999999999991,"5min":8.2739999999999991,"15min":8.2739999999999991},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":8.3320000000000007,"5min":8.3320000000000007,"15min":8.3320000000000007},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":8.0299999999999994,"5min":8.0299999999999994,"15min":8.0299999999999994},"last":0.47199999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.3030000000000008,"5min":8.3030000000000008,"15min":8.3030000000000008},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":8.3800000000000008,"5min":8.3800000000000008,"15min":8.3800000000000008},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":8.3580000000000005,"5min":8.3580000000000005,"15min":8.3580000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":8.0839999999999996,"5min":8.0839999999999996,"15min":8.0839999999999996},"last":0.55700000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":409600,"data_stored":171646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.246},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.27800000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":5.0899999999999999,"5min":5.0899999999999999,"15min":5.0899999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.316}]},{"osd":4,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.501}]},{"osd":6,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.51000000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":172258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":4.359,"5min":4.359,"15min":4.359},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.78300000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.82499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":167255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.57999999999999996}]},{"osd":1,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.498}]},{"osd":2,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.439}]},{"osd":3,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.375},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48799999999999999}]},{"osd":7,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60099999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":173007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40300000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.57599999999999996}]},{"osd":6,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.61199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:37:23.883 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:37:24.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.998+0000 7f0914cc6700 1 Processor -- start 2023-06-23T18:37:24.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.998+0000 7f0914cc6700 1 -- start start 2023-06-23T18:37:24.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.999+0000 7f0914cc6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09100cdad0 0x7f09100ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:24.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:23.999+0000 7f0914cc6700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f09100ce620 0x7f09100cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:24.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.000+0000 7f0914cc6700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091015cd50 con 0x7f0910159170 2023-06-23T18:37:24.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.000+0000 7f0914cc6700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09100cea80 con 0x7f09100ce620 2023-06-23T18:37:24.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.000+0000 7f0914cc6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091015cfa0 con 0x7f09100cdad0 2023-06-23T18:37:24.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.000+0000 7f090ed9d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f09100ce620 0x7f09100cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:24.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.000+0000 7f090dd9b700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09100cdad0 0x7f09100ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:24.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.000+0000 7f090ed9d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f09100ce620 0x7f09100cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:48484/0 (socket says 172.21.15.79:48484) 2023-06-23T18:37:24.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.000+0000 7f090e59c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f0910159170 0x7f09100cd3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:53564/0 (socket says 172.21.15.79:53564) 2023-06-23T18:37:24.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.000+0000 7f090ed9d700 1 -- 172.21.15.79:0/4277177546 learned_addr learned my addr 172.21.15.79:0/4277177546 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:24.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.000+0000 7f090ed9d700 1 -- 172.21.15.79:0/4277177546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09100cdad0 msgr2=0x7f09100ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:24.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.000+0000 7f090ed9d700 1 --2- 172.21.15.79:0/4277177546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09100cdad0 0x7f09100ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:24.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.000+0000 7f090ed9d700 1 -- 172.21.15.79:0/4277177546 >> v1:172.21.15.155:6789/0 conn(0x7f0910159170 legacy=0x7f09100cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:24.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.000+0000 7f090ed9d700 1 -- 172.21.15.79:0/4277177546 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091015d280 con 0x7f09100ce620 2023-06-23T18:37:24.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.000+0000 7f090dd9b700 1 --2- 172.21.15.79:0/4277177546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09100cdad0 0x7f09100ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:24.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.000+0000 7f090ed9d700 1 --2- 172.21.15.79:0/4277177546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09100ce620 0x7f09100cc1d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0908009e50 tx=0x7f090800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fcc11abde08e475 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:24.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.001+0000 7f090d59a700 1 -- 172.21.15.79:0/4277177546 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090800e070 con 0x7f09100ce620 2023-06-23T18:37:24.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.001+0000 7f090d59a700 1 -- 172.21.15.79:0/4277177546 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090800b910 con 0x7f09100ce620 2023-06-23T18:37:24.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.001+0000 7f090d59a700 1 -- 172.21.15.79:0/4277177546 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090800bcc0 con 0x7f09100ce620 2023-06-23T18:37:24.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.001+0000 7f0914cc6700 1 -- 172.21.15.79:0/4277177546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09100ce620 msgr2=0x7f09100cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:24.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.001+0000 7f0914cc6700 1 --2- 172.21.15.79:0/4277177546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09100ce620 0x7f09100cc1d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0908009e50 tx=0x7f090800b330 comp rx=0 tx=0).stop 2023-06-23T18:37:24.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.001+0000 7f0914cc6700 1 -- 172.21.15.79:0/4277177546 shutdown_connections 2023-06-23T18:37:24.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.001+0000 7f0914cc6700 1 --2- 172.21.15.79:0/4277177546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09100ce620 0x7f09100cc1d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:24.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.001+0000 7f0914cc6700 1 --2- 172.21.15.79:0/4277177546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09100cdad0 0x7f09100ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:24.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.001+0000 7f0914cc6700 1 -- 172.21.15.79:0/4277177546 >> 172.21.15.79:0/4277177546 conn(0x7f091006d660 msgr2=0x7f091006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:24.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.001+0000 7f0914cc6700 1 -- 172.21.15.79:0/4277177546 shutdown_connections 2023-06-23T18:37:24.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.001+0000 7f0914cc6700 1 -- 172.21.15.79:0/4277177546 wait complete. 2023-06-23T18:37:24.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.003+0000 7f0914cc6700 1 Processor -- start 2023-06-23T18:37:24.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.003+0000 7f0914cc6700 1 -- start start 2023-06-23T18:37:24.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f0914cc6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09100cdad0 0x7f09100b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:24.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f0914cc6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09100ce620 0x7f09100b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:24.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f0914cc6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0910159170 0x7f09100c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:24.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f0914cc6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09101562b0 con 0x7f0910159170 2023-06-23T18:37:24.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f0914cc6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09100b53a0 con 0x7f09100ce620 2023-06-23T18:37:24.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f0914cc6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09101e29e0 con 0x7f09100cdad0 2023-06-23T18:37:24.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f090ed9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0910159170 0x7f09100c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:24.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f090ed9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0910159170 0x7f09100c68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42264/0 (socket says 172.21.15.79:42264) 2023-06-23T18:37:24.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f090ed9d700 1 -- 172.21.15.79:0/2999200835 learned_addr learned my addr 172.21.15.79:0/2999200835 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:24.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f090dd9b700 1 --2- 172.21.15.79:0/2999200835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09100ce620 0x7f09100b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:24.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f090e59c700 1 --2- 172.21.15.79:0/2999200835 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09100cdad0 0x7f09100b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:24.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f090ed9d700 1 -- 172.21.15.79:0/2999200835 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09100cdad0 msgr2=0x7f09100b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:24.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f090ed9d700 1 --2- 172.21.15.79:0/2999200835 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09100cdad0 0x7f09100b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:24.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f090ed9d700 1 -- 172.21.15.79:0/2999200835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09100ce620 msgr2=0x7f09100b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:24.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f090ed9d700 1 --2- 172.21.15.79:0/2999200835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09100ce620 0x7f09100b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:24.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.004+0000 7f090ed9d700 1 -- 172.21.15.79:0/2999200835 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09000094a0 con 0x7f0910159170 2023-06-23T18:37:24.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.005+0000 7f090ed9d700 1 --2- 172.21.15.79:0/2999200835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0910159170 0x7f09100c68c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0910070bc0 tx=0x7f0908004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:24.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.005+0000 7f08feffd700 1 -- 172.21.15.79:0/2999200835 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908004960 con 0x7f0910159170 2023-06-23T18:37:24.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.005+0000 7f08feffd700 1 -- 172.21.15.79:0/2999200835 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09080099b0 con 0x7f0910159170 2023-06-23T18:37:24.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.005+0000 7f0914cc6700 1 -- 172.21.15.79:0/2999200835 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0908009b30 con 0x7f0910159170 2023-06-23T18:37:24.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.005+0000 7f08feffd700 1 -- 172.21.15.79:0/2999200835 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090801c680 con 0x7f0910159170 2023-06-23T18:37:24.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.005+0000 7f0914cc6700 1 -- 172.21.15.79:0/2999200835 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09101e2fa0 con 0x7f0910159170 2023-06-23T18:37:24.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.006+0000 7f0914cc6700 1 -- 172.21.15.79:0/2999200835 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09100b5770 con 0x7f0910159170 2023-06-23T18:37:24.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.006+0000 7f08feffd700 1 -- 172.21.15.79:0/2999200835 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f090801c7e0 con 0x7f0910159170 2023-06-23T18:37:24.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.007+0000 7f08feffd700 1 --2- 172.21.15.79:0/2999200835 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f08f006ace0 0x7f08f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:24.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.007+0000 7f08feffd700 1 -- 172.21.15.79:0/2999200835 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f090808ae50 con 0x7f0910159170 2023-06-23T18:37:24.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.007+0000 7f090e59c700 1 --2- 172.21.15.79:0/2999200835 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f08f006ace0 0x7f08f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:24.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.007+0000 7f090e59c700 1 --2- 172.21.15.79:0/2999200835 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f08f006ace0 0x7f08f006d190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0904000f60 tx=0x7f0904007d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:24.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.011+0000 7f08feffd700 1 -- 172.21.15.79:0/2999200835 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0908056290 con 0x7f0910159170 2023-06-23T18:37:24.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.202+0000 7f0914cc6700 1 -- 172.21.15.79:0/2999200835 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09100cee90 con 0x7f0910159170 2023-06-23T18:37:24.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.203+0000 7f08feffd700 1 -- 172.21.15.79:0/2999200835 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+14722 (secure 0 0 0) 0x7f090805f090 con 0x7f0910159170 2023-06-23T18:37:24.209 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:24.209 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":37,"fsid":"c6d43825-8b0e-40e8-8d7f-a5ab36592233","created":"2023-06-23T18:35:35.171397+0000","modified":"2023-06-23T18:37:20.982895+0000","last_up_change":"2023-06-23T18:35:45.927394+0000","last_in_change":"2023-06-23T18:35:37.676106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T18:35:46.467546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T18:35:46.735662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2932a84-07f5-4a23-9b19-7fd0e288a1ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6809","nonce":1517442919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6811","nonce":1517442919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6815","nonce":1517442919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1517442919},{"type":"v1","addr":"172.21.15.79:6813","nonce":1517442919}]},"public_addr":"172.21.15.79:6809/1517442919","cluster_addr":"172.21.15.79:6811/1517442919","heartbeat_back_addr":"172.21.15.79:6815/1517442919","heartbeat_front_addr":"172.21.15.79:6813/1517442919","state":["exists","up"]},{"osd":1,"uuid":"e6b1bc10-2bc9-4af2-9833-fade6110194d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6801","nonce":2920568506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6803","nonce":2920568506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6807","nonce":2920568506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2920568506},{"type":"v1","addr":"172.21.15.79:6805","nonce":2920568506}]},"public_addr":"172.21.15.79:6801/2920568506","cluster_addr":"172.21.15.79:6803/2920568506","heartbeat_back_addr":"172.21.15.79:6807/2920568506","heartbeat_front_addr":"172.21.15.79:6805/2920568506","state":["exists","up"]},{"osd":2,"uuid":"5b16c204-3aa9-4741-89d4-f8c1530b399b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6817","nonce":1735841648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6819","nonce":1735841648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6823","nonce":1735841648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1735841648},{"type":"v1","addr":"172.21.15.79:6821","nonce":1735841648}]},"public_addr":"172.21.15.79:6817/1735841648","cluster_addr":"172.21.15.79:6819/1735841648","heartbeat_back_addr":"172.21.15.79:6823/1735841648","heartbeat_front_addr":"172.21.15.79:6821/1735841648","state":["exists","up"]},{"osd":3,"uuid":"d2efa121-6af5-4b4d-b2df-2df7070cc986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6825","nonce":1549244775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6827","nonce":1549244775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6831","nonce":1549244775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1549244775},{"type":"v1","addr":"172.21.15.79:6829","nonce":1549244775}]},"public_addr":"172.21.15.79:6825/1549244775","cluster_addr":"172.21.15.79:6827/1549244775","heartbeat_back_addr":"172.21.15.79:6831/1549244775","heartbeat_front_addr":"172.21.15.79:6829/1549244775","state":["exists","up"]},{"osd":4,"uuid":"5ac17afb-4981-4b93-aaf5-8e0224b3d408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6817","nonce":2160815572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6819","nonce":2160815572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6823","nonce":2160815572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2160815572},{"type":"v1","addr":"172.21.15.155:6821","nonce":2160815572}]},"public_addr":"172.21.15.155:6817/2160815572","cluster_addr":"172.21.15.155:6819/2160815572","heartbeat_back_addr":"172.21.15.155:6823/2160815572","heartbeat_front_addr":"172.21.15.155:6821/2160815572","state":["exists","up"]},{"osd":5,"uuid":"e602dbcd-be18-4aa0-a3f6-8bff41056208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6803","nonce":4051222998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6807","nonce":4051222998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6815","nonce":4051222998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":4051222998},{"type":"v1","addr":"172.21.15.155:6811","nonce":4051222998}]},"public_addr":"172.21.15.155:6803/4051222998","cluster_addr":"172.21.15.155:6807/4051222998","heartbeat_back_addr":"172.21.15.155:6815/4051222998","heartbeat_front_addr":"172.21.15.155:6811/4051222998","state":["exists","up"]},{"osd":6,"uuid":"dfcc906c-8979-4f61-b01a-88540da6a253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6802","nonce":1282622313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6806","nonce":1282622313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6814","nonce":1282622313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1282622313},{"type":"v1","addr":"172.21.15.155:6810","nonce":1282622313}]},"public_addr":"172.21.15.155:6802/1282622313","cluster_addr":"172.21.15.155:6806/1282622313","heartbeat_back_addr":"172.21.15.155:6814/1282622313","heartbeat_front_addr":"172.21.15.155:6810/1282622313","state":["exists","up"]},{"osd":7,"uuid":"5858f908-e534-4ab8-9b82-ff6b01ff58b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6825","nonce":3369626517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6827","nonce":3369626517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6831","nonce":3369626517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3369626517},{"type":"v1","addr":"172.21.15.155:6829","nonce":3369626517}]},"public_addr":"172.21.15.155:6825/3369626517","cluster_addr":"172.21.15.155:6827/3369626517","heartbeat_back_addr":"172.21.15.155:6831/3369626517","heartbeat_front_addr":"172.21.15.155:6829/3369626517","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.811181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.543730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:44.985000+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.833864+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.836611+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.825199+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.812492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:35:42.880029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6836/1030786659":"2023-06-24T18:37:17.177503+0000","172.21.15.79:6834/1030786659":"2023-06-24T18:37:17.177503+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:37:24.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.205+0000 7f0914cc6700 1 -- 172.21.15.79:0/2999200835 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f08f006ace0 msgr2=0x7f08f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:24.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.205+0000 7f0914cc6700 1 --2- 172.21.15.79:0/2999200835 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f08f006ace0 0x7f08f006d190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0904000f60 tx=0x7f0904007d10 comp rx=0 tx=0).stop 2023-06-23T18:37:24.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.205+0000 7f0914cc6700 1 -- 172.21.15.79:0/2999200835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0910159170 msgr2=0x7f09100c68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:24.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.205+0000 7f0914cc6700 1 --2- 172.21.15.79:0/2999200835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0910159170 0x7f09100c68c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0910070bc0 tx=0x7f0908004ad0 comp rx=0 tx=0).stop 2023-06-23T18:37:24.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.205+0000 7f0914cc6700 1 -- 172.21.15.79:0/2999200835 shutdown_connections 2023-06-23T18:37:24.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.205+0000 7f0914cc6700 1 --2- 172.21.15.79:0/2999200835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0910159170 0x7f09100c68c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:24.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.205+0000 7f0914cc6700 1 --2- 172.21.15.79:0/2999200835 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09100cdad0 0x7f09100b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:24.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.205+0000 7f0914cc6700 1 --2- 172.21.15.79:0/2999200835 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f08f006ace0 0x7f08f006d190 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:24.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.205+0000 7f0914cc6700 1 --2- 172.21.15.79:0/2999200835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09100ce620 0x7f09100b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:24.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.205+0000 7f0914cc6700 1 -- 172.21.15.79:0/2999200835 >> 172.21.15.79:0/2999200835 conn(0x7f091006d660 msgr2=0x7f09100cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:24.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.205+0000 7f0914cc6700 1 -- 172.21.15.79:0/2999200835 shutdown_connections 2023-06-23T18:37:24.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:24.205+0000 7f0914cc6700 1 -- 172.21.15.79:0/2999200835 wait complete. 2023-06-23T18:37:25.223 INFO:tasks.ceph:Scrubbing osd.0 2023-06-23T18:37:25.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-06-23T18:37:25.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.356+0000 7fe5a647c700 1 Processor -- start 2023-06-23T18:37:25.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.356+0000 7fe5a647c700 1 -- start start 2023-06-23T18:37:25.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.357+0000 7fe5a647c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a015c710 0x7fe5a00b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.357+0000 7fe5a647c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe5a00aed90 0x7fe5a00b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.357+0000 7fe5a647c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a015d620 con 0x7fe5a00cd410 2023-06-23T18:37:25.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.357+0000 7fe5a647c700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a00ca870 con 0x7fe5a00aed90 2023-06-23T18:37:25.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.357+0000 7fe5a647c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00724a0 con 0x7fe5a015c710 2023-06-23T18:37:25.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.357+0000 7fe5a4a19700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe5a00cd410 0x7fe5a006eeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45314/0 (socket says 172.21.15.79:45314) 2023-06-23T18:37:25.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.357+0000 7fe5a4a19700 1 -- 172.21.15.79:0/1325863617 learned_addr learned my addr 172.21.15.79:0/1325863617 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:25.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.357+0000 7fe59f7fe700 1 --2- 172.21.15.79:0/1325863617 >> v2:172.21.15.155:3300/0 conn(0x7fe5a00aed90 0x7fe5a00b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.357+0000 7fe59ffff700 1 --2- 172.21.15.79:0/1325863617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a015c710 0x7fe5a00b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.357+0000 7fe59effd700 1 -- 172.21.15.79:0/1325863617 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777057286 0 0) 0x7fe5a015d620 con 0x7fe5a00cd410 2023-06-23T18:37:25.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.357+0000 7fe59effd700 1 -- 172.21.15.79:0/1325863617 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5880035b0 con 0x7fe5a00cd410 2023-06-23T18:37:25.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.358+0000 7fe59ffff700 1 -- 172.21.15.79:0/1325863617 >> v2:172.21.15.155:3300/0 conn(0x7fe5a00aed90 msgr2=0x7fe5a00b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:25.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.358+0000 7fe59ffff700 1 --2- 172.21.15.79:0/1325863617 >> v2:172.21.15.155:3300/0 conn(0x7fe5a00aed90 0x7fe5a00b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.358+0000 7fe59ffff700 1 -- 172.21.15.79:0/1325863617 >> v1:172.21.15.79:6789/0 conn(0x7fe5a00cd410 legacy=0x7fe5a006eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.358+0000 7fe59effd700 1 -- 172.21.15.79:0/1325863617 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 721292302 0 0) 0x7fe5880035b0 con 0x7fe5a00cd410 2023-06-23T18:37:25.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.358+0000 7fe59ffff700 1 -- 172.21.15.79:0/1325863617 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a00cd9d0 con 0x7fe5a015c710 2023-06-23T18:37:25.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.358+0000 7fe59f7fe700 1 --2- 172.21.15.79:0/1325863617 >> v2:172.21.15.155:3300/0 conn(0x7fe5a00aed90 0x7fe5a00b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:25.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.358+0000 7fe59ffff700 1 --2- 172.21.15.79:0/1325863617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a015c710 0x7fe5a00b12c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe594009840 tx=0x7fe594009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a89d89555ad8900 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:25.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.358+0000 7fe59effd700 1 -- 172.21.15.79:0/1325863617 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5980019f0 con 0x7fe5a015c710 2023-06-23T18:37:25.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.358+0000 7fe59effd700 1 -- 172.21.15.79:0/1325863617 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe594004030 con 0x7fe5a015c710 2023-06-23T18:37:25.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.358+0000 7fe59effd700 1 -- 172.21.15.79:0/1325863617 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5940043c0 con 0x7fe5a015c710 2023-06-23T18:37:25.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.359+0000 7fe5a647c700 1 -- 172.21.15.79:0/1325863617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a015c710 msgr2=0x7fe5a00b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.359+0000 7fe5a647c700 1 --2- 172.21.15.79:0/1325863617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a015c710 0x7fe5a00b12c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe594009840 tx=0x7fe594009b50 comp rx=0 tx=0).stop 2023-06-23T18:37:25.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.359+0000 7fe5a647c700 1 -- 172.21.15.79:0/1325863617 shutdown_connections 2023-06-23T18:37:25.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.359+0000 7fe5a647c700 1 --2- 172.21.15.79:0/1325863617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a015c710 0x7fe5a00b12c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.359+0000 7fe5a647c700 1 --2- 172.21.15.79:0/1325863617 >> v2:172.21.15.155:3300/0 conn(0x7fe5a00aed90 0x7fe5a00b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.359+0000 7fe5a647c700 1 -- 172.21.15.79:0/1325863617 >> 172.21.15.79:0/1325863617 conn(0x7fe5a006e200 msgr2=0x7fe5a00cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:25.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.359+0000 7fe5a647c700 1 -- 172.21.15.79:0/1325863617 shutdown_connections 2023-06-23T18:37:25.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.359+0000 7fe5a647c700 1 -- 172.21.15.79:0/1325863617 wait complete. 2023-06-23T18:37:25.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.360+0000 7fe5a647c700 1 Processor -- start 2023-06-23T18:37:25.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.360+0000 7fe5a647c700 1 -- start start 2023-06-23T18:37:25.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe5a647c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a00aed90 0x7fe5a014c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe5a647c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5a00cd410 0x7fe5a014cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe5a647c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5a014fb20 0x7fe5a0150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe5a647c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a014cb70 con 0x7fe5a00cd410 2023-06-23T18:37:25.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe5a647c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a0158770 con 0x7fe5a014fb20 2023-06-23T18:37:25.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe5a647c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a0158a50 con 0x7fe5a00aed90 2023-06-23T18:37:25.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe59f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5a00cd410 0x7fe5a014cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe59f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5a00cd410 0x7fe5a014cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42276/0 (socket says 172.21.15.79:42276) 2023-06-23T18:37:25.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe59f7fe700 1 -- 172.21.15.79:0/1475510140 learned_addr learned my addr 172.21.15.79:0/1475510140 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:25.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe5a4a19700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5a014fb20 0x7fe5a0150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe59ffff700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a00aed90 0x7fe5a014c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe59f7fe700 1 -- 172.21.15.79:0/1475510140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a00aed90 msgr2=0x7fe5a014c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe59f7fe700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a00aed90 0x7fe5a014c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe59f7fe700 1 -- 172.21.15.79:0/1475510140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5a014fb20 msgr2=0x7fe5a0150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe59f7fe700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5a014fb20 0x7fe5a0150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe59f7fe700 1 -- 172.21.15.79:0/1475510140 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5900094a0 con 0x7fe5a00cd410 2023-06-23T18:37:25.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe5a4a19700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5a014fb20 0x7fe5a0150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:25.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.361+0000 7fe59f7fe700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5a00cd410 0x7fe5a014cd10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe59000ebe0 tx=0x7fe590009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:25.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.362+0000 7fe59cff9700 1 -- 172.21.15.79:0/1475510140 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59000c910 con 0x7fe5a00cd410 2023-06-23T18:37:25.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.362+0000 7fe59cff9700 1 -- 172.21.15.79:0/1475510140 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59000ca70 con 0x7fe5a00cd410 2023-06-23T18:37:25.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.362+0000 7fe5a647c700 1 -- 172.21.15.79:0/1475510140 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5940094a0 con 0x7fe5a00cd410 2023-06-23T18:37:25.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.362+0000 7fe59cff9700 1 -- 172.21.15.79:0/1475510140 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59000cde0 con 0x7fe5a00cd410 2023-06-23T18:37:25.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.362+0000 7fe5a647c700 1 -- 172.21.15.79:0/1475510140 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a0159010 con 0x7fe5a00cd410 2023-06-23T18:37:25.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.362+0000 7fe5a647c700 1 -- 172.21.15.79:0/1475510140 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe5a015cfd0 con 0x7fe5a00cd410 2023-06-23T18:37:25.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.363+0000 7fe59cff9700 1 -- 172.21.15.79:0/1475510140 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe590013470 con 0x7fe5a00cd410 2023-06-23T18:37:25.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.363+0000 7fe59cff9700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe58c06ace0 0x7fe58c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.364+0000 7fe59cff9700 1 -- 172.21.15.79:0/1475510140 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe590089350 con 0x7fe5a00cd410 2023-06-23T18:37:25.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.364+0000 7fe59ffff700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe58c06ace0 0x7fe58c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.364+0000 7fe59cff9700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.79:6808/1517442919,v1:172.21.15.79:6809/1517442919] conn(0x7fe58c071150 0x7fe58c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.364+0000 7fe59cff9700 1 -- 172.21.15.79:0/1475510140 --> [v2:172.21.15.79:6808/1517442919,v1:172.21.15.79:6809/1517442919] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe58c073c30 con 0x7fe58c071150 2023-06-23T18:37:25.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.364+0000 7fe59cff9700 1 -- 172.21.15.79:0/1475510140 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe59008b9f0 con 0x7fe5a00cd410 2023-06-23T18:37:25.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.364+0000 7fe5a4a19700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.79:6808/1517442919,v1:172.21.15.79:6809/1517442919] conn(0x7fe58c071150 0x7fe58c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.364+0000 7fe59ffff700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe58c06ace0 0x7fe58c06d190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00d0940 tx=0x7fe594008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:25.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.364+0000 7fe5a4a19700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.79:6808/1517442919,v1:172.21.15.79:6809/1517442919] conn(0x7fe58c071150 0x7fe58c073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:25.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.365+0000 7fe59cff9700 1 -- 172.21.15.79:0/1475510140 <== osd.0 v2:172.21.15.79:6808/1517442919 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe58c073c30 con 0x7fe58c071150 2023-06-23T18:37:25.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.379+0000 7fe5a647c700 1 -- 172.21.15.79:0/1475510140 --> [v2:172.21.15.79:6808/1517442919,v1:172.21.15.79:6809/1517442919] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe5a015ccf0 con 0x7fe58c071150 2023-06-23T18:37:25.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.379+0000 7fe59cff9700 1 -- 172.21.15.79:0/1475510140 <== osd.0 v2:172.21.15.79:6808/1517442919 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe5980072f0 con 0x7fe58c071150 2023-06-23T18:37:25.384 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-23T18:37:25.385 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-23T18:37:25.385 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-23T18:37:25.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.379+0000 7fe5a647c700 1 -- 172.21.15.79:0/1475510140 >> [v2:172.21.15.79:6808/1517442919,v1:172.21.15.79:6809/1517442919] conn(0x7fe58c071150 msgr2=0x7fe58c073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.379+0000 7fe5a647c700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.79:6808/1517442919,v1:172.21.15.79:6809/1517442919] conn(0x7fe58c071150 0x7fe58c073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.379+0000 7fe5a647c700 1 -- 172.21.15.79:0/1475510140 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe58c06ace0 msgr2=0x7fe58c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.379+0000 7fe5a647c700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fe58c06ace0 0x7fe58c06d190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00d0940 tx=0x7fe594008030 comp rx=0 tx=0).stop 2023-06-23T18:37:25.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.379+0000 7fe5a647c700 1 -- 172.21.15.79:0/1475510140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5a00cd410 msgr2=0x7fe5a014cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.379+0000 7fe5a647c700 1 --2- 172.21.15.79:0/1475510140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5a00cd410 0x7fe5a014cd10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe59000ebe0 tx=0x7fe590009ec0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.379+0000 7fe5a4a19700 1 -- 172.21.15.79:0/1475510140 reap_dead start 2023-06-23T18:37:25.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.379+0000 7fe5a647c700 1 -- 172.21.15.79:0/1475510140 shutdown_connections 2023-06-23T18:37:25.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.379+0000 7fe5a647c700 1 -- 172.21.15.79:0/1475510140 >> 172.21.15.79:0/1475510140 conn(0x7fe5a006e200 msgr2=0x7fe5a00b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:25.388 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.380+0000 7fe5a647c700 1 -- 172.21.15.79:0/1475510140 shutdown_connections 2023-06-23T18:37:25.388 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.380+0000 7fe5a647c700 1 -- 172.21.15.79:0/1475510140 wait complete. 2023-06-23T18:37:25.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-06-23T18:37:25.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.517+0000 7f481056f700 1 Processor -- start 2023-06-23T18:37:25.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.517+0000 7f481056f700 1 -- start start 2023-06-23T18:37:25.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.518+0000 7f481056f700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f480815a380 0x7f48080ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.518+0000 7f481056f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480815a750 0x7f48080cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.518+0000 7f481056f700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48080b1d40 con 0x7f48080ade50 2023-06-23T18:37:25.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.518+0000 7f481056f700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48080b2020 con 0x7f480815a380 2023-06-23T18:37:25.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.518+0000 7f481056f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48080ad650 con 0x7f480815a750 2023-06-23T18:37:25.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.518+0000 7f480db0a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480815a750 0x7f48080cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.518+0000 7f480e30b700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f480815a380 0x7f48080ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.518+0000 7f480eb0c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f48080ade50 0x7f48080ae290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:53598/0 (socket says 172.21.15.79:53598) 2023-06-23T18:37:25.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.518+0000 7f480eb0c700 1 -- 172.21.15.79:0/217529522 learned_addr learned my addr 172.21.15.79:0/217529522 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:25.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.518+0000 7f480db0a700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480815a750 0x7f48080cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:35054/0 (socket says 172.21.15.79:35054) 2023-06-23T18:37:25.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.518+0000 7f480e30b700 1 -- 172.21.15.79:0/217529522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480815a750 msgr2=0x7f48080cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:25.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.518+0000 7f480e30b700 1 --2- 172.21.15.79:0/217529522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480815a750 0x7f48080cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.518+0000 7f480e30b700 1 -- 172.21.15.79:0/217529522 >> v1:172.21.15.155:6789/0 conn(0x7f48080ade50 legacy=0x7f48080ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.518+0000 7f480d309700 1 -- 172.21.15.79:0/217529522 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002775995 0 0) 0x7f48080b1d40 con 0x7f48080ade50 2023-06-23T18:37:25.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.519+0000 7f480e30b700 1 -- 172.21.15.79:0/217529522 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48080ad930 con 0x7f480815a380 2023-06-23T18:37:25.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.519+0000 7f480e30b700 1 --2- 172.21.15.79:0/217529522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f480815a380 0x7f48080ce4f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4804009820 tx=0x7f4804009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=833dd11c91e259e7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:25.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.519+0000 7f480d309700 1 -- 172.21.15.79:0/217529522 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804005e30 con 0x7f480815a380 2023-06-23T18:37:25.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.519+0000 7f480d309700 1 -- 172.21.15.79:0/217529522 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4804006030 con 0x7f480815a380 2023-06-23T18:37:25.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.519+0000 7f480d309700 1 -- 172.21.15.79:0/217529522 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48040063c0 con 0x7f480815a380 2023-06-23T18:37:25.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.519+0000 7f481056f700 1 -- 172.21.15.79:0/217529522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f480815a380 msgr2=0x7f48080ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.519+0000 7f481056f700 1 --2- 172.21.15.79:0/217529522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f480815a380 0x7f48080ce4f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4804009820 tx=0x7f4804009b30 comp rx=0 tx=0).stop 2023-06-23T18:37:25.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.519+0000 7f481056f700 1 -- 172.21.15.79:0/217529522 shutdown_connections 2023-06-23T18:37:25.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.519+0000 7f481056f700 1 --2- 172.21.15.79:0/217529522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f480815a380 0x7f48080ce4f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.519+0000 7f481056f700 1 --2- 172.21.15.79:0/217529522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480815a750 0x7f48080cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.519+0000 7f481056f700 1 -- 172.21.15.79:0/217529522 >> 172.21.15.79:0/217529522 conn(0x7f4808089fc0 msgr2=0x7f4808099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:25.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.519+0000 7f481056f700 1 -- 172.21.15.79:0/217529522 shutdown_connections 2023-06-23T18:37:25.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.520+0000 7f481056f700 1 -- 172.21.15.79:0/217529522 wait complete. 2023-06-23T18:37:25.534 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.521+0000 7f481056f700 1 Processor -- start 2023-06-23T18:37:25.534 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.521+0000 7f481056f700 1 -- start start 2023-06-23T18:37:25.534 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.521+0000 7f481056f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48080ade50 0x7f48081f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.534 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.521+0000 7f481056f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480815a380 0x7f480806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.521+0000 7f481056f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f480815a750 0x7f480806f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.521+0000 7f481056f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4808146610 con 0x7f480815a750 2023-06-23T18:37:25.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.521+0000 7f481056f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480815bbc0 con 0x7f48080ade50 2023-06-23T18:37:25.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.521+0000 7f481056f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480815cbb0 con 0x7f480815a380 2023-06-23T18:37:25.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f480eb0c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f480815a750 0x7f480806f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f480eb0c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f480815a750 0x7f480806f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42294/0 (socket says 172.21.15.79:42294) 2023-06-23T18:37:25.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f480eb0c700 1 -- 172.21.15.79:0/3625702 learned_addr learned my addr 172.21.15.79:0/3625702 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:25.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f480e30b700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48080ade50 0x7f48081f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f480db0a700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480815a380 0x7f480806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f480eb0c700 1 -- 172.21.15.79:0/3625702 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480815a380 msgr2=0x7f480806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f480eb0c700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480815a380 0x7f480806ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f480eb0c700 1 -- 172.21.15.79:0/3625702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48080ade50 msgr2=0x7f48081f2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f480eb0c700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48080ade50 0x7f48081f2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f480eb0c700 1 -- 172.21.15.79:0/3625702 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4804009430 con 0x7f480815a750 2023-06-23T18:37:25.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f480e30b700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48080ade50 0x7f48081f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:25.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f480eb0c700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f480815a750 0x7f480806f180 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f480000cc70 tx=0x7f480000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:25.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f47feffd700 1 -- 172.21.15.79:0/3625702 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480000ada0 con 0x7f480815a750 2023-06-23T18:37:25.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f47feffd700 1 -- 172.21.15.79:0/3625702 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480000ce70 con 0x7f480815a750 2023-06-23T18:37:25.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f481056f700 1 -- 172.21.15.79:0/3625702 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480815be10 con 0x7f480815a750 2023-06-23T18:37:25.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f47feffd700 1 -- 172.21.15.79:0/3625702 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48000072f0 con 0x7f480815a750 2023-06-23T18:37:25.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.522+0000 7f481056f700 1 -- 172.21.15.79:0/3625702 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480815c3d0 con 0x7f480815a750 2023-06-23T18:37:25.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.523+0000 7f481056f700 1 -- 172.21.15.79:0/3625702 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480815ca20 con 0x7f480815a750 2023-06-23T18:37:25.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.524+0000 7f47feffd700 1 -- 172.21.15.79:0/3625702 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4800007450 con 0x7f480815a750 2023-06-23T18:37:25.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.524+0000 7f47feffd700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f47ec06ace0 0x7f47ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.524+0000 7f47feffd700 1 -- 172.21.15.79:0/3625702 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4800088ff0 con 0x7f480815a750 2023-06-23T18:37:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.524+0000 7f480e30b700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f47ec06ace0 0x7f47ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.524+0000 7f480e30b700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f47ec06ace0 0x7f47ec06d190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4804000c00 tx=0x7f4804008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.528+0000 7f47feffd700 1 -- 172.21.15.79:0/3625702 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4800054430 con 0x7f480815a750 2023-06-23T18:37:25.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.714+0000 7f481056f700 1 -- 172.21.15.79:0/3625702 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f480804e240 con 0x7f47ec06ace0 2023-06-23T18:37:25.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.715+0000 7f47feffd700 1 -- 172.21.15.79:0/3625702 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f480804e240 con 0x7f47ec06ace0 2023-06-23T18:37:25.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.716+0000 7f481056f700 1 -- 172.21.15.79:0/3625702 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f47ec06ace0 msgr2=0x7f47ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.716+0000 7f481056f700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f47ec06ace0 0x7f47ec06d190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4804000c00 tx=0x7f4804008070 comp rx=0 tx=0).stop 2023-06-23T18:37:25.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.717+0000 7f481056f700 1 -- 172.21.15.79:0/3625702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f480815a750 msgr2=0x7f480806f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.717+0000 7f481056f700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f480815a750 0x7f480806f180 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f480000cc70 tx=0x7f480000a340 comp rx=0 tx=0).stop 2023-06-23T18:37:25.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.717+0000 7f481056f700 1 -- 172.21.15.79:0/3625702 shutdown_connections 2023-06-23T18:37:25.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.717+0000 7f481056f700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f480815a750 0x7f480806f180 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.717+0000 7f481056f700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f47ec06ace0 0x7f47ec06d190 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.717+0000 7f481056f700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48080ade50 0x7f48081f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.717+0000 7f481056f700 1 --2- 172.21.15.79:0/3625702 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480815a380 0x7f480806ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.723 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.717+0000 7f481056f700 1 -- 172.21.15.79:0/3625702 >> 172.21.15.79:0/3625702 conn(0x7f4808089fc0 msgr2=0x7f4808158da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:25.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.717+0000 7f481056f700 1 -- 172.21.15.79:0/3625702 shutdown_connections 2023-06-23T18:37:25.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.717+0000 7f481056f700 1 -- 172.21.15.79:0/3625702 wait complete. 2023-06-23T18:37:25.724 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 0 to deep-scrub 2023-06-23T18:37:25.733 INFO:tasks.ceph:Scrubbing osd.1 2023-06-23T18:37:25.734 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-06-23T18:37:25.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.852+0000 7f6469186700 1 Processor -- start 2023-06-23T18:37:25.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.852+0000 7f6469186700 1 -- start start 2023-06-23T18:37:25.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f6469186700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f646415a0a0 0x7f64640ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f6469186700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f646415e490 0x7f64640cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f6469186700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646415eab0 con 0x7f64640cea60 2023-06-23T18:37:25.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f6469186700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646415ed90 con 0x7f646415e490 2023-06-23T18:37:25.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f6469186700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64640cef00 con 0x7f646415a0a0 2023-06-23T18:37:25.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f646359e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f646415e490 0x7f64640cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f646359e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f646415e490 0x7f64640cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42302/0 (socket says 172.21.15.79:42302) 2023-06-23T18:37:25.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f646359e700 1 -- 172.21.15.79:0/4002987312 learned_addr learned my addr 172.21.15.79:0/4002987312 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:25.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f6462d9d700 1 --2- 172.21.15.79:0/4002987312 >> v2:172.21.15.155:3300/0 conn(0x7f646415a0a0 0x7f64640ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f646359e700 1 -- 172.21.15.79:0/4002987312 >> v2:172.21.15.155:3300/0 conn(0x7f646415a0a0 msgr2=0x7f64640ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:25.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f646359e700 1 --2- 172.21.15.79:0/4002987312 >> v2:172.21.15.155:3300/0 conn(0x7f646415a0a0 0x7f64640ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f646359e700 1 -- 172.21.15.79:0/4002987312 >> v1:172.21.15.155:6789/0 conn(0x7f64640cea60 legacy=0x7f646415dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:25.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f646359e700 1 -- 172.21.15.79:0/4002987312 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64640cb140 con 0x7f646415e490 2023-06-23T18:37:25.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f6462d9d700 1 --2- 172.21.15.79:0/4002987312 >> v2:172.21.15.155:3300/0 conn(0x7f646415a0a0 0x7f64640ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:25.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f646359e700 1 --2- 172.21.15.79:0/4002987312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f646415e490 0x7f64640cbc20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f64640b3510 tx=0x7f645c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b320da16971b52ef server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:25.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f6461d9b700 1 -- 172.21.15.79:0/4002987312 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c00e070 con 0x7f646415e490 2023-06-23T18:37:25.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f6461d9b700 1 -- 172.21.15.79:0/4002987312 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c003a20 con 0x7f646415e490 2023-06-23T18:37:25.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.853+0000 7f6461d9b700 1 -- 172.21.15.79:0/4002987312 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c003dd0 con 0x7f646415e490 2023-06-23T18:37:25.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.854+0000 7f6469186700 1 -- 172.21.15.79:0/4002987312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f646415e490 msgr2=0x7f64640cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.854+0000 7f6469186700 1 --2- 172.21.15.79:0/4002987312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f646415e490 0x7f64640cbc20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f64640b3510 tx=0x7f645c00b330 comp rx=0 tx=0).stop 2023-06-23T18:37:25.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.854+0000 7f6469186700 1 -- 172.21.15.79:0/4002987312 shutdown_connections 2023-06-23T18:37:25.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.854+0000 7f6469186700 1 --2- 172.21.15.79:0/4002987312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f646415e490 0x7f64640cbc20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.854+0000 7f6469186700 1 --2- 172.21.15.79:0/4002987312 >> v2:172.21.15.155:3300/0 conn(0x7f646415a0a0 0x7f64640ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.854+0000 7f6469186700 1 -- 172.21.15.79:0/4002987312 >> 172.21.15.79:0/4002987312 conn(0x7f6464099cc0 msgr2=0x7f6464089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:25.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.854+0000 7f6469186700 1 -- 172.21.15.79:0/4002987312 shutdown_connections 2023-06-23T18:37:25.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.854+0000 7f6469186700 1 -- 172.21.15.79:0/4002987312 wait complete. 2023-06-23T18:37:25.873 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.855+0000 7f6469186700 1 Processor -- start 2023-06-23T18:37:25.873 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.856+0000 7f6469186700 1 -- start start 2023-06-23T18:37:25.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.856+0000 7f6469186700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64640cea60 0x7f64641569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.856+0000 7f6469186700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f646415a0a0 0x7f64640cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.856+0000 7f6469186700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f646415e490 0x7f64640cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.856+0000 7f6469186700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6464157200 con 0x7f646415a0a0 2023-06-23T18:37:25.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.856+0000 7f6469186700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64640ccd60 con 0x7f64640cea60 2023-06-23T18:37:25.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.856+0000 7f6469186700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64640cd040 con 0x7f646415e490 2023-06-23T18:37:25.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.856+0000 7f646259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f646415a0a0 0x7f64640cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.856+0000 7f646259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f646415a0a0 0x7f64640cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42306/0 (socket says 172.21.15.79:42306) 2023-06-23T18:37:25.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.856+0000 7f646259c700 1 -- 172.21.15.79:0/1200714234 learned_addr learned my addr 172.21.15.79:0/1200714234 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:25.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.856+0000 7f6462d9d700 1 --2- 172.21.15.79:0/1200714234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64640cea60 0x7f64641569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.856+0000 7f646359e700 1 --2- 172.21.15.79:0/1200714234 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f646415e490 0x7f64640cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.857+0000 7f646259c700 1 -- 172.21.15.79:0/1200714234 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f646415e490 msgr2=0x7f64640cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.857+0000 7f646259c700 1 --2- 172.21.15.79:0/1200714234 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f646415e490 0x7f64640cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.857+0000 7f646259c700 1 -- 172.21.15.79:0/1200714234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64640cea60 msgr2=0x7f64641569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.857+0000 7f646259c700 1 --2- 172.21.15.79:0/1200714234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64640cea60 0x7f64641569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.857+0000 7f646259c700 1 -- 172.21.15.79:0/1200714234 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c009b30 con 0x7f646415a0a0 2023-06-23T18:37:25.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.857+0000 7f646259c700 1 --2- 172.21.15.79:0/1200714234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f646415a0a0 0x7f64640cde00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f645400b4a0 tx=0x7f645400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:25.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.857+0000 7f644b7fe700 1 -- 172.21.15.79:0/1200714234 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454011830 con 0x7f646415a0a0 2023-06-23T18:37:25.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.857+0000 7f644b7fe700 1 -- 172.21.15.79:0/1200714234 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6454011990 con 0x7f646415a0a0 2023-06-23T18:37:25.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.857+0000 7f6469186700 1 -- 172.21.15.79:0/1200714234 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64640cd320 con 0x7f646415a0a0 2023-06-23T18:37:25.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.857+0000 7f644b7fe700 1 -- 172.21.15.79:0/1200714234 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454011d40 con 0x7f646415a0a0 2023-06-23T18:37:25.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.857+0000 7f6469186700 1 -- 172.21.15.79:0/1200714234 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64640cd8e0 con 0x7f646415a0a0 2023-06-23T18:37:25.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.858+0000 7f6469186700 1 -- 172.21.15.79:0/1200714234 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64640cb420 con 0x7f646415a0a0 2023-06-23T18:37:25.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.858+0000 7f644b7fe700 1 -- 172.21.15.79:0/1200714234 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f645401a400 con 0x7f646415a0a0 2023-06-23T18:37:25.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.859+0000 7f644b7fe700 1 --2- 172.21.15.79:0/1200714234 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f645006ace0 0x7f645006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.859+0000 7f644b7fe700 1 -- 172.21.15.79:0/1200714234 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6454088d80 con 0x7f646415a0a0 2023-06-23T18:37:25.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.859+0000 7f6462d9d700 1 --2- 172.21.15.79:0/1200714234 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f645006ace0 0x7f645006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.859+0000 7f644b7fe700 1 --2- 172.21.15.79:0/1200714234 >> [v2:172.21.15.79:6800/2920568506,v1:172.21.15.79:6801/2920568506] conn(0x7f6450071150 0x7f6450073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:25.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.859+0000 7f644b7fe700 1 -- 172.21.15.79:0/1200714234 --> [v2:172.21.15.79:6800/2920568506,v1:172.21.15.79:6801/2920568506] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6450073c30 con 0x7f6450071150 2023-06-23T18:37:25.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.859+0000 7f644b7fe700 1 -- 172.21.15.79:0/1200714234 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f645408d030 con 0x7f646415a0a0 2023-06-23T18:37:25.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.859+0000 7f646359e700 1 --2- 172.21.15.79:0/1200714234 >> [v2:172.21.15.79:6800/2920568506,v1:172.21.15.79:6801/2920568506] conn(0x7f6450071150 0x7f6450073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:25.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.859+0000 7f6462d9d700 1 --2- 172.21.15.79:0/1200714234 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f645006ace0 0x7f645006d190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f64640c6f10 tx=0x7f6458008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:25.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.859+0000 7f646359e700 1 --2- 172.21.15.79:0/1200714234 >> [v2:172.21.15.79:6800/2920568506,v1:172.21.15.79:6801/2920568506] conn(0x7f6450071150 0x7f6450073560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:25.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.860+0000 7f644b7fe700 1 -- 172.21.15.79:0/1200714234 <== osd.1 v2:172.21.15.79:6800/2920568506 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6450073c30 con 0x7f6450071150 2023-06-23T18:37:25.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.874+0000 7f6469186700 1 -- 172.21.15.79:0/1200714234 --> [v2:172.21.15.79:6800/2920568506,v1:172.21.15.79:6801/2920568506] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f646415e890 con 0x7f6450071150 2023-06-23T18:37:25.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.874+0000 7f644b7fe700 1 -- 172.21.15.79:0/1200714234 <== osd.1 v2:172.21.15.79:6800/2920568506 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f645c0094f0 con 0x7f6450071150 2023-06-23T18:37:25.887 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-23T18:37:25.887 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-23T18:37:25.887 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-23T18:37:25.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.874+0000 7f6469186700 1 -- 172.21.15.79:0/1200714234 >> [v2:172.21.15.79:6800/2920568506,v1:172.21.15.79:6801/2920568506] conn(0x7f6450071150 msgr2=0x7f6450073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.874+0000 7f6469186700 1 --2- 172.21.15.79:0/1200714234 >> [v2:172.21.15.79:6800/2920568506,v1:172.21.15.79:6801/2920568506] conn(0x7f6450071150 0x7f6450073560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.874+0000 7f6469186700 1 -- 172.21.15.79:0/1200714234 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f645006ace0 msgr2=0x7f645006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.874+0000 7f6469186700 1 --2- 172.21.15.79:0/1200714234 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f645006ace0 0x7f645006d190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f64640c6f10 tx=0x7f6458008040 comp rx=0 tx=0).stop 2023-06-23T18:37:25.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.874+0000 7f6469186700 1 -- 172.21.15.79:0/1200714234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f646415a0a0 msgr2=0x7f64640cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:25.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.874+0000 7f6469186700 1 --2- 172.21.15.79:0/1200714234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f646415a0a0 0x7f64640cde00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f645400b4a0 tx=0x7f645400b7b0 comp rx=0 tx=0).stop 2023-06-23T18:37:25.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.875+0000 7f646359e700 1 -- 172.21.15.79:0/1200714234 reap_dead start 2023-06-23T18:37:25.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.875+0000 7f6469186700 1 -- 172.21.15.79:0/1200714234 shutdown_connections 2023-06-23T18:37:25.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.875+0000 7f6469186700 1 -- 172.21.15.79:0/1200714234 >> 172.21.15.79:0/1200714234 conn(0x7f6464099cc0 msgr2=0x7f64640ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:25.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.875+0000 7f6469186700 1 -- 172.21.15.79:0/1200714234 shutdown_connections 2023-06-23T18:37:25.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:25.875+0000 7f6469186700 1 -- 172.21.15.79:0/1200714234 wait complete. 2023-06-23T18:37:25.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-06-23T18:37:26.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.008+0000 7f36b3f3c700 1 Processor -- start 2023-06-23T18:37:26.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.009+0000 7f36b3f3c700 1 -- start start 2023-06-23T18:37:26.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.009+0000 7f36b3f3c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f36ac15a380 0x7f36ac0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.009+0000 7f36b3f3c700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f36ac0cea30 0x7f36ac0b1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.009+0000 7f36b3f3c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac0cee20 con 0x7f36ac15a750 2023-06-23T18:37:26.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.009+0000 7f36b3f3c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ac0b1d40 con 0x7f36ac15a380 2023-06-23T18:37:26.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.009+0000 7f36b3f3c700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ac0b2020 con 0x7f36ac0cea30 2023-06-23T18:37:26.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.010+0000 7f36b1cd8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f36ac15a380 0x7f36ac0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.010+0000 7f36b1cd8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f36ac15a380 0x7f36ac0ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42314/0 (socket says 172.21.15.79:42314) 2023-06-23T18:37:26.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.010+0000 7f36b1cd8700 1 -- 172.21.15.79:0/566895927 learned_addr learned my addr 172.21.15.79:0/566895927 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:26.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.010+0000 7f36b24d9700 1 --2- 172.21.15.79:0/566895927 >> v2:172.21.15.155:3300/0 conn(0x7f36ac0cea30 0x7f36ac0b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.010+0000 7f36b1cd8700 1 -- 172.21.15.79:0/566895927 >> v2:172.21.15.155:3300/0 conn(0x7f36ac0cea30 msgr2=0x7f36ac0b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:26.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.010+0000 7f36b1cd8700 1 --2- 172.21.15.79:0/566895927 >> v2:172.21.15.155:3300/0 conn(0x7f36ac0cea30 0x7f36ac0b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.010+0000 7f36b1cd8700 1 -- 172.21.15.79:0/566895927 >> v1:172.21.15.155:6789/0 conn(0x7f36ac15a750 legacy=0x7f36ac0ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:26.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.010+0000 7f36b1cd8700 1 -- 172.21.15.79:0/566895927 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac0ad650 con 0x7f36ac15a380 2023-06-23T18:37:26.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.010+0000 7f36b1cd8700 1 --2- 172.21.15.79:0/566895927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ac15a380 0x7f36ac0ce4f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0c43f0 tx=0x7f369c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a1d4d15eb074d6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:26.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.010+0000 7f36b0cd6700 1 -- 172.21.15.79:0/566895927 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c005e30 con 0x7f36ac15a380 2023-06-23T18:37:26.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.010+0000 7f36b0cd6700 1 -- 172.21.15.79:0/566895927 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c006030 con 0x7f36ac15a380 2023-06-23T18:37:26.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.010+0000 7f36b0cd6700 1 -- 172.21.15.79:0/566895927 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c0063e0 con 0x7f36ac15a380 2023-06-23T18:37:26.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.011+0000 7f36b3f3c700 1 -- 172.21.15.79:0/566895927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ac15a380 msgr2=0x7f36ac0ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.011+0000 7f36b3f3c700 1 --2- 172.21.15.79:0/566895927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ac15a380 0x7f36ac0ce4f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0c43f0 tx=0x7f369c009bd0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.011+0000 7f36b3f3c700 1 -- 172.21.15.79:0/566895927 shutdown_connections 2023-06-23T18:37:26.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.011+0000 7f36b3f3c700 1 --2- 172.21.15.79:0/566895927 >> v2:172.21.15.155:3300/0 conn(0x7f36ac0cea30 0x7f36ac0b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.011+0000 7f36b3f3c700 1 --2- 172.21.15.79:0/566895927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ac15a380 0x7f36ac0ce4f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.011+0000 7f36b3f3c700 1 -- 172.21.15.79:0/566895927 >> 172.21.15.79:0/566895927 conn(0x7f36ac089fc0 msgr2=0x7f36ac099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:26.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.011+0000 7f36b3f3c700 1 -- 172.21.15.79:0/566895927 shutdown_connections 2023-06-23T18:37:26.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.011+0000 7f36b3f3c700 1 -- 172.21.15.79:0/566895927 wait complete. 2023-06-23T18:37:26.024 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.012+0000 7f36b3f3c700 1 Processor -- start 2023-06-23T18:37:26.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.012+0000 7f36b3f3c700 1 -- start start 2023-06-23T18:37:26.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b3f3c700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36ac0cea30 0x7f36ac150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b3f3c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ac15a380 0x7f36ac06bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b3f3c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ac15a750 0x7f36ac15cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b3f3c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac151400 con 0x7f36ac15a750 2023-06-23T18:37:26.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b3f3c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac15c860 con 0x7f36ac15a380 2023-06-23T18:37:26.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b3f3c700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac0c5a70 con 0x7f36ac0cea30 2023-06-23T18:37:26.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b24d9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ac15a750 0x7f36ac15cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b24d9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ac15a750 0x7f36ac15cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42324/0 (socket says 172.21.15.79:42324) 2023-06-23T18:37:26.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b24d9700 1 -- 172.21.15.79:0/3852285940 learned_addr learned my addr 172.21.15.79:0/3852285940 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b14d7700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ac15a380 0x7f36ac06bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b1cd8700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36ac0cea30 0x7f36ac150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b24d9700 1 -- 172.21.15.79:0/3852285940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36ac0cea30 msgr2=0x7f36ac150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b24d9700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36ac0cea30 0x7f36ac150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b24d9700 1 -- 172.21.15.79:0/3852285940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ac15a380 msgr2=0x7f36ac06bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b24d9700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ac15a380 0x7f36ac06bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b24d9700 1 -- 172.21.15.79:0/3852285940 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c009500 con 0x7f36ac15a750 2023-06-23T18:37:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b1cd8700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36ac0cea30 0x7f36ac150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b24d9700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ac15a750 0x7f36ac15cb70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f36a400cd70 tx=0x7f36a400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36a27fc700 1 -- 172.21.15.79:0/3852285940 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4003a20 con 0x7f36ac15a750 2023-06-23T18:37:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36a27fc700 1 -- 172.21.15.79:0/3852285940 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a4003b80 con 0x7f36ac15a750 2023-06-23T18:37:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36b3f3c700 1 -- 172.21.15.79:0/3852285940 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ac0c5460 con 0x7f36ac15a750 2023-06-23T18:37:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.013+0000 7f36a27fc700 1 -- 172.21.15.79:0/3852285940 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4004020 con 0x7f36ac15a750 2023-06-23T18:37:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.014+0000 7f36b3f3c700 1 -- 172.21.15.79:0/3852285940 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ac15c580 con 0x7f36ac15a750 2023-06-23T18:37:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.014+0000 7f36b3f3c700 1 -- 172.21.15.79:0/3852285940 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ac0b1a60 con 0x7f36ac15a750 2023-06-23T18:37:26.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.015+0000 7f36a27fc700 1 -- 172.21.15.79:0/3852285940 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f36a4004180 con 0x7f36ac15a750 2023-06-23T18:37:26.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.015+0000 7f36a27fc700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f369006ace0 0x7f369006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.015+0000 7f36a27fc700 1 -- 172.21.15.79:0/3852285940 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f36a4012070 con 0x7f36ac15a750 2023-06-23T18:37:26.032 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.015+0000 7f36b1cd8700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f369006ace0 0x7f369006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.033 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.016+0000 7f36b1cd8700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f369006ace0 0x7f369006d190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0c43f0 tx=0x7f369c008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:26.033 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.020+0000 7f36a27fc700 1 -- 172.21.15.79:0/3852285940 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f36a4054d30 con 0x7f36ac15a750 2023-06-23T18:37:26.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.206+0000 7f36b3f3c700 1 -- 172.21.15.79:0/3852285940 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f36ac0ad930 con 0x7f369006ace0 2023-06-23T18:37:26.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.206+0000 7f36a27fc700 1 -- 172.21.15.79:0/3852285940 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f369c0041c0 con 0x7f369006ace0 2023-06-23T18:37:26.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.208+0000 7f36b3f3c700 1 -- 172.21.15.79:0/3852285940 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f369006ace0 msgr2=0x7f369006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.208+0000 7f36b3f3c700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f369006ace0 0x7f369006d190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0c43f0 tx=0x7f369c008070 comp rx=0 tx=0).stop 2023-06-23T18:37:26.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.208+0000 7f36b3f3c700 1 -- 172.21.15.79:0/3852285940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ac15a750 msgr2=0x7f36ac15cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.208+0000 7f36b3f3c700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ac15a750 0x7f36ac15cb70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f36a400cd70 tx=0x7f36a400a570 comp rx=0 tx=0).stop 2023-06-23T18:37:26.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.208+0000 7f36b3f3c700 1 -- 172.21.15.79:0/3852285940 shutdown_connections 2023-06-23T18:37:26.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.208+0000 7f36b3f3c700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ac15a750 0x7f36ac15cb70 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.208+0000 7f36b3f3c700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36ac0cea30 0x7f36ac150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.208+0000 7f36b3f3c700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f369006ace0 0x7f369006d190 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.208+0000 7f36b3f3c700 1 --2- 172.21.15.79:0/3852285940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ac15a380 0x7f36ac06bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.208+0000 7f36b3f3c700 1 -- 172.21.15.79:0/3852285940 >> 172.21.15.79:0/3852285940 conn(0x7f36ac089fc0 msgr2=0x7f36ac06a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:26.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.208+0000 7f36b3f3c700 1 -- 172.21.15.79:0/3852285940 shutdown_connections 2023-06-23T18:37:26.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.208+0000 7f36b3f3c700 1 -- 172.21.15.79:0/3852285940 wait complete. 2023-06-23T18:37:26.216 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 1 to deep-scrub 2023-06-23T18:37:26.225 INFO:tasks.ceph:Scrubbing osd.2 2023-06-23T18:37:26.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-06-23T18:37:26.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.345+0000 7f9a9801d700 1 Processor -- start 2023-06-23T18:37:26.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.346+0000 7f9a9801d700 1 -- start start 2023-06-23T18:37:26.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.346+0000 7f9a9801d700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9a90160ac0 0x7f9a900cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.346+0000 7f9a9801d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a900ceef0 con 0x7f9a900cfe20 2023-06-23T18:37:26.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.346+0000 7f9a9801d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a900cf450 con 0x7f9a900d0ea0 2023-06-23T18:37:26.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.346+0000 7f9a9801d700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a900cf120 con 0x7f9a90160ac0 2023-06-23T18:37:26.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.346+0000 7f9a965ba700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9a900cfe20 0x7f9a9015fee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45374/0 (socket says 172.21.15.79:45374) 2023-06-23T18:37:26.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.346+0000 7f9a965ba700 1 -- 172.21.15.79:0/4058639802 learned_addr learned my addr 172.21.15.79:0/4058639802 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:26.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a955b8700 1 --2- 172.21.15.79:0/4058639802 >> v2:172.21.15.155:3300/0 conn(0x7f9a90160ac0 0x7f9a900cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 -- 172.21.15.79:0/4058639802 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681266514 0 0) 0x7f9a900ceef0 con 0x7f9a900cfe20 2023-06-23T18:37:26.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 -- 172.21.15.79:0/4058639802 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a7c0035b0 con 0x7f9a900cfe20 2023-06-23T18:37:26.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 -- 172.21.15.79:0/4058639802 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 687335525 0 0) 0x7f9a7c0035b0 con 0x7f9a900cfe20 2023-06-23T18:37:26.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 -- 172.21.15.79:0/4058639802 >> v2:172.21.15.155:3300/0 conn(0x7f9a90160ac0 msgr2=0x7f9a900cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:26.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 --2- 172.21.15.79:0/4058639802 >> v2:172.21.15.155:3300/0 conn(0x7f9a90160ac0 0x7f9a900cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 -- 172.21.15.79:0/4058639802 >> v1:172.21.15.155:6789/0 conn(0x7f9a900d0ea0 legacy=0x7f9a900d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 -- 172.21.15.79:0/4058639802 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a900cd420 con 0x7f9a900cfe20 2023-06-23T18:37:26.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 -- 172.21.15.79:0/4058639802 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f9a880019f0 con 0x7f9a900cfe20 2023-06-23T18:37:26.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 -- 172.21.15.79:0/4058639802 >> v1:172.21.15.79:6789/0 conn(0x7f9a900cfe20 legacy=0x7f9a9015fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 --2- 172.21.15.79:0/4058639802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a7c003a00 0x7f9a7c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 --2- 172.21.15.79:0/4058639802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a7c004850 0x7f9a7c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 --2- 172.21.15.79:0/4058639802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a7c005320 0x7f9a7c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 -- 172.21.15.79:0/4058639802 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900cf120 con 0x7f9a7c005320 2023-06-23T18:37:26.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 -- 172.21.15.79:0/4058639802 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c00c250 con 0x7f9a7c003a00 2023-06-23T18:37:26.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a94db7700 1 -- 172.21.15.79:0/4058639802 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c00c410 con 0x7f9a7c004850 2023-06-23T18:37:26.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a955b8700 1 --2- 172.21.15.79:0/4058639802 >> v2:172.21.15.155:3300/0 conn(0x7f9a90160ac0 0x7f9a900cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:26.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a965ba700 1 --2- 172.21.15.79:0/4058639802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a7c005320 0x7f9a7c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a95db9700 1 --2- 172.21.15.79:0/4058639802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a7c003a00 0x7f9a7c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a955b8700 1 --2- 172.21.15.79:0/4058639802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a7c004850 0x7f9a7c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a965ba700 1 -- 172.21.15.79:0/4058639802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a7c004850 msgr2=0x7f9a7c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:26.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a965ba700 1 --2- 172.21.15.79:0/4058639802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a7c004850 0x7f9a7c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a965ba700 1 -- 172.21.15.79:0/4058639802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a7c003a00 msgr2=0x7f9a7c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:26.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.347+0000 7f9a965ba700 1 --2- 172.21.15.79:0/4058639802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a7c003a00 0x7f9a7c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.348+0000 7f9a965ba700 1 -- 172.21.15.79:0/4058639802 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c0035b0 con 0x7f9a7c005320 2023-06-23T18:37:26.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.348+0000 7f9a95db9700 1 --2- 172.21.15.79:0/4058639802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a7c003a00 0x7f9a7c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:26.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.348+0000 7f9a965ba700 1 -- 172.21.15.79:0/4058639802 reap_dead start 2023-06-23T18:37:26.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.348+0000 7f9a965ba700 1 --2- 172.21.15.79:0/4058639802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a7c005320 0x7f9a7c00ba10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800a800 tx=0x7f9a7c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db59a0ef45468538 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:26.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.348+0000 7f9a94db7700 1 -- 172.21.15.79:0/4058639802 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8800a2f0 con 0x7f9a7c005320 2023-06-23T18:37:26.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.348+0000 7f9a94db7700 1 -- 172.21.15.79:0/4058639802 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a88012620 con 0x7f9a7c005320 2023-06-23T18:37:26.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.348+0000 7f9a94db7700 1 -- 172.21.15.79:0/4058639802 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a880129d0 con 0x7f9a7c005320 2023-06-23T18:37:26.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.348+0000 7f9a9801d700 1 -- 172.21.15.79:0/4058639802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a7c005320 msgr2=0x7f9a7c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.348+0000 7f9a9801d700 1 --2- 172.21.15.79:0/4058639802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a7c005320 0x7f9a7c00ba10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800a800 tx=0x7f9a7c005880 comp rx=0 tx=0).stop 2023-06-23T18:37:26.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.348+0000 7f9a9801d700 1 -- 172.21.15.79:0/4058639802 shutdown_connections 2023-06-23T18:37:26.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.348+0000 7f9a9801d700 1 --2- 172.21.15.79:0/4058639802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a7c005320 0x7f9a7c00ba10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.348+0000 7f9a9801d700 1 -- 172.21.15.79:0/4058639802 >> 172.21.15.79:0/4058639802 conn(0x7f9a900715c0 msgr2=0x7f9a90071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:26.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.348+0000 7f9a9801d700 1 -- 172.21.15.79:0/4058639802 shutdown_connections 2023-06-23T18:37:26.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.348+0000 7f9a9801d700 1 -- 172.21.15.79:0/4058639802 wait complete. 2023-06-23T18:37:26.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.350+0000 7f9a9801d700 1 Processor -- start 2023-06-23T18:37:26.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.350+0000 7f9a9801d700 1 -- start start 2023-06-23T18:37:26.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.350+0000 7f9a9801d700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a7c005320 0x7f9a900ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.350+0000 7f9a9801d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a900b68c0 0x7f9a900c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.350+0000 7f9a9801d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a900c84f0 0x7f9a901f1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.350+0000 7f9a9801d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900c8990 con 0x7f9a900c84f0 2023-06-23T18:37:26.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.350+0000 7f9a9801d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900b7030 con 0x7f9a900b68c0 2023-06-23T18:37:26.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.350+0000 7f9a9801d700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900b7310 con 0x7f9a7c005320 2023-06-23T18:37:26.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.350+0000 7f9a965ba700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a900c84f0 0x7f9a901f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.350+0000 7f9a965ba700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a900c84f0 0x7f9a901f1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42344/0 (socket says 172.21.15.79:42344) 2023-06-23T18:37:26.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.350+0000 7f9a965ba700 1 -- 172.21.15.79:0/518347960 learned_addr learned my addr 172.21.15.79:0/518347960 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:26.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.350+0000 7f9a95db9700 1 --2- 172.21.15.79:0/518347960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a7c005320 0x7f9a900ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.350+0000 7f9a955b8700 1 --2- 172.21.15.79:0/518347960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a900b68c0 0x7f9a900c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.351+0000 7f9a965ba700 1 -- 172.21.15.79:0/518347960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a7c005320 msgr2=0x7f9a900ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.351+0000 7f9a965ba700 1 --2- 172.21.15.79:0/518347960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a7c005320 0x7f9a900ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.351+0000 7f9a965ba700 1 -- 172.21.15.79:0/518347960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a900b68c0 msgr2=0x7f9a900c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.351+0000 7f9a965ba700 1 --2- 172.21.15.79:0/518347960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a900b68c0 0x7f9a900c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.351+0000 7f9a965ba700 1 -- 172.21.15.79:0/518347960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a900cb560 con 0x7f9a900c84f0 2023-06-23T18:37:26.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.351+0000 7f9a965ba700 1 --2- 172.21.15.79:0/518347960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a900c84f0 0x7f9a901f1500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9a9008be70 tx=0x7f9a88001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:26.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.351+0000 7f9a867fc700 1 -- 172.21.15.79:0/518347960 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a880019f0 con 0x7f9a900c84f0 2023-06-23T18:37:26.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.351+0000 7f9a867fc700 1 -- 172.21.15.79:0/518347960 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a88001250 con 0x7f9a900c84f0 2023-06-23T18:37:26.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.351+0000 7f9a9801d700 1 -- 172.21.15.79:0/518347960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a900cb840 con 0x7f9a900c84f0 2023-06-23T18:37:26.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.351+0000 7f9a867fc700 1 -- 172.21.15.79:0/518347960 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8801d640 con 0x7f9a900c84f0 2023-06-23T18:37:26.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.351+0000 7f9a9801d700 1 -- 172.21.15.79:0/518347960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a900b6d30 con 0x7f9a900c84f0 2023-06-23T18:37:26.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.352+0000 7f9a9801d700 1 -- 172.21.15.79:0/518347960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a900cd9e0 con 0x7f9a900c84f0 2023-06-23T18:37:26.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.352+0000 7f9a867fc700 1 -- 172.21.15.79:0/518347960 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a8801d7a0 con 0x7f9a900c84f0 2023-06-23T18:37:26.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.353+0000 7f9a867fc700 1 --2- 172.21.15.79:0/518347960 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9a7406ace0 0x7f9a7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.353+0000 7f9a867fc700 1 -- 172.21.15.79:0/518347960 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9a8808baf0 con 0x7f9a900c84f0 2023-06-23T18:37:26.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.353+0000 7f9a95db9700 1 --2- 172.21.15.79:0/518347960 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9a7406ace0 0x7f9a7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.370 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.353+0000 7f9a867fc700 1 --2- 172.21.15.79:0/518347960 >> [v2:172.21.15.79:6816/1735841648,v1:172.21.15.79:6817/1735841648] conn(0x7f9a74071150 0x7f9a74073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.353+0000 7f9a867fc700 1 -- 172.21.15.79:0/518347960 --> [v2:172.21.15.79:6816/1735841648,v1:172.21.15.79:6817/1735841648] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a74073c30 con 0x7f9a74071150 2023-06-23T18:37:26.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.353+0000 7f9a867fc700 1 -- 172.21.15.79:0/518347960 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a8808bf20 con 0x7f9a900c84f0 2023-06-23T18:37:26.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.353+0000 7f9a955b8700 1 --2- 172.21.15.79:0/518347960 >> [v2:172.21.15.79:6816/1735841648,v1:172.21.15.79:6817/1735841648] conn(0x7f9a74071150 0x7f9a74073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.353+0000 7f9a95db9700 1 --2- 172.21.15.79:0/518347960 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9a7406ace0 0x7f9a7406d190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9a80007fd0 tx=0x7f9a80009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:26.371 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.353+0000 7f9a955b8700 1 --2- 172.21.15.79:0/518347960 >> [v2:172.21.15.79:6816/1735841648,v1:172.21.15.79:6817/1735841648] conn(0x7f9a74071150 0x7f9a74073560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:26.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.354+0000 7f9a867fc700 1 -- 172.21.15.79:0/518347960 <== osd.2 v2:172.21.15.79:6816/1735841648 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9a74073c30 con 0x7f9a74071150 2023-06-23T18:37:26.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.368+0000 7f9a9801d700 1 -- 172.21.15.79:0/518347960 --> [v2:172.21.15.79:6816/1735841648,v1:172.21.15.79:6817/1735841648] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9a900cd700 con 0x7f9a74071150 2023-06-23T18:37:26.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.368+0000 7f9a867fc700 1 -- 172.21.15.79:0/518347960 <== osd.2 v2:172.21.15.79:6816/1735841648 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9a8c009790 con 0x7f9a74071150 2023-06-23T18:37:26.373 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-23T18:37:26.374 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-23T18:37:26.374 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-23T18:37:26.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.368+0000 7f9a9801d700 1 -- 172.21.15.79:0/518347960 >> [v2:172.21.15.79:6816/1735841648,v1:172.21.15.79:6817/1735841648] conn(0x7f9a74071150 msgr2=0x7f9a74073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.368+0000 7f9a9801d700 1 --2- 172.21.15.79:0/518347960 >> [v2:172.21.15.79:6816/1735841648,v1:172.21.15.79:6817/1735841648] conn(0x7f9a74071150 0x7f9a74073560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.369+0000 7f9a9801d700 1 -- 172.21.15.79:0/518347960 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9a7406ace0 msgr2=0x7f9a7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.369+0000 7f9a9801d700 1 --2- 172.21.15.79:0/518347960 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f9a7406ace0 0x7f9a7406d190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9a80007fd0 tx=0x7f9a80009040 comp rx=0 tx=0).stop 2023-06-23T18:37:26.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.369+0000 7f9a9801d700 1 -- 172.21.15.79:0/518347960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a900c84f0 msgr2=0x7f9a901f1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.369+0000 7f9a9801d700 1 --2- 172.21.15.79:0/518347960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a900c84f0 0x7f9a901f1500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9a9008be70 tx=0x7f9a88001d80 comp rx=0 tx=0).stop 2023-06-23T18:37:26.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.369+0000 7f9a965ba700 1 -- 172.21.15.79:0/518347960 reap_dead start 2023-06-23T18:37:26.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.369+0000 7f9a9801d700 1 -- 172.21.15.79:0/518347960 shutdown_connections 2023-06-23T18:37:26.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.369+0000 7f9a9801d700 1 -- 172.21.15.79:0/518347960 >> 172.21.15.79:0/518347960 conn(0x7f9a900715c0 msgr2=0x7f9a9015da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:26.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.369+0000 7f9a9801d700 1 -- 172.21.15.79:0/518347960 shutdown_connections 2023-06-23T18:37:26.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.369+0000 7f9a9801d700 1 -- 172.21.15.79:0/518347960 wait complete. 2023-06-23T18:37:26.385 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-06-23T18:37:26.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.501+0000 7f4d7a88f700 1 Processor -- start 2023-06-23T18:37:26.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.501+0000 7f4d7a88f700 1 -- start start 2023-06-23T18:37:26.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d7a88f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d74158830 0x7f4d740c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d7a88f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4d7406a5a0 0x7f4d740c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d7a88f700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d740c3f80 con 0x7f4d740c5280 2023-06-23T18:37:26.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d7a88f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d740c4260 con 0x7f4d7406a5a0 2023-06-23T18:37:26.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d7a88f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740c4540 con 0x7f4d74158830 2023-06-23T18:37:26.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d78e2c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4d7406a5a0 0x7f4d740c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d737fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f4d740c5280 0x7f4d74069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45394/0 (socket says 172.21.15.79:45394) 2023-06-23T18:37:26.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d737fe700 1 -- 172.21.15.79:0/3648731395 learned_addr learned my addr 172.21.15.79:0/3648731395 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:26.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d78e2c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4d7406a5a0 0x7f4d740c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42354/0 (socket says 172.21.15.79:42354) 2023-06-23T18:37:26.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d73fff700 1 --2- 172.21.15.79:0/3648731395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d74158830 0x7f4d740c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d78e2c700 1 -- 172.21.15.79:0/3648731395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d74158830 msgr2=0x7f4d740c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:26.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d78e2c700 1 --2- 172.21.15.79:0/3648731395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d74158830 0x7f4d740c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d78e2c700 1 -- 172.21.15.79:0/3648731395 >> v1:172.21.15.79:6789/0 conn(0x7f4d740c5280 legacy=0x7f4d74069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d78e2c700 1 -- 172.21.15.79:0/3648731395 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d740c4820 con 0x7f4d7406a5a0 2023-06-23T18:37:26.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d72ffd700 1 -- 172.21.15.79:0/3648731395 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750966963 0 0) 0x7f4d740c3f80 con 0x7f4d740c5280 2023-06-23T18:37:26.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.502+0000 7f4d73fff700 1 --2- 172.21.15.79:0/3648731395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d74158830 0x7f4d740c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:26.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.503+0000 7f4d78e2c700 1 --2- 172.21.15.79:0/3648731395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d7406a5a0 0x7f4d740c92c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009d90 tx=0x7f4d6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec024c0f65d3a3c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:26.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.503+0000 7f4d72ffd700 1 -- 172.21.15.79:0/3648731395 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c00e070 con 0x7f4d7406a5a0 2023-06-23T18:37:26.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.503+0000 7f4d72ffd700 1 -- 172.21.15.79:0/3648731395 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c00bbd0 con 0x7f4d7406a5a0 2023-06-23T18:37:26.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.503+0000 7f4d72ffd700 1 -- 172.21.15.79:0/3648731395 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c012430 con 0x7f4d7406a5a0 2023-06-23T18:37:26.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.503+0000 7f4d7a88f700 1 -- 172.21.15.79:0/3648731395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d7406a5a0 msgr2=0x7f4d740c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.503+0000 7f4d7a88f700 1 --2- 172.21.15.79:0/3648731395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d7406a5a0 0x7f4d740c92c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009d90 tx=0x7f4d6c00b330 comp rx=0 tx=0).stop 2023-06-23T18:37:26.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.503+0000 7f4d7a88f700 1 -- 172.21.15.79:0/3648731395 shutdown_connections 2023-06-23T18:37:26.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.503+0000 7f4d7a88f700 1 --2- 172.21.15.79:0/3648731395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d7406a5a0 0x7f4d740c92c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.503+0000 7f4d7a88f700 1 --2- 172.21.15.79:0/3648731395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d74158830 0x7f4d740c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.503+0000 7f4d7a88f700 1 -- 172.21.15.79:0/3648731395 >> 172.21.15.79:0/3648731395 conn(0x7f4d740cf3d0 msgr2=0x7f4d740cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:26.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.503+0000 7f4d7a88f700 1 -- 172.21.15.79:0/3648731395 shutdown_connections 2023-06-23T18:37:26.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.503+0000 7f4d7a88f700 1 -- 172.21.15.79:0/3648731395 wait complete. 2023-06-23T18:37:26.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.504+0000 7f4d7a88f700 1 Processor -- start 2023-06-23T18:37:26.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.505+0000 7f4d7a88f700 1 -- start start 2023-06-23T18:37:26.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.505+0000 7f4d7a88f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d7406a5a0 0x7f4d7414e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.505+0000 7f4d7a88f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d740c5280 0x7f4d74145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.505+0000 7f4d7a88f700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d74158830 0x7f4d7414ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.505+0000 7f4d7a88f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7414dfa0 con 0x7f4d7406a5a0 2023-06-23T18:37:26.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.505+0000 7f4d7a88f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7414e900 con 0x7f4d740c5280 2023-06-23T18:37:26.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.505+0000 7f4d7a88f700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d74141ad0 con 0x7f4d74158830 2023-06-23T18:37:26.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d73fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d7406a5a0 0x7f4d7414e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d73fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d7406a5a0 0x7f4d7414e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42356/0 (socket says 172.21.15.79:42356) 2023-06-23T18:37:26.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d73fff700 1 -- 172.21.15.79:0/499124373 learned_addr learned my addr 172.21.15.79:0/499124373 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:26.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d737fe700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d740c5280 0x7f4d74145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d78e2c700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d74158830 0x7f4d7414ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d73fff700 1 -- 172.21.15.79:0/499124373 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d74158830 msgr2=0x7f4d7414ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d73fff700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d74158830 0x7f4d7414ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d73fff700 1 -- 172.21.15.79:0/499124373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d740c5280 msgr2=0x7f4d74145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d73fff700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d740c5280 0x7f4d74145fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d73fff700 1 -- 172.21.15.79:0/499124373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c009a70 con 0x7f4d7406a5a0 2023-06-23T18:37:26.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d737fe700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d740c5280 0x7f4d74145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:26.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d78e2c700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d74158830 0x7f4d7414ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:26.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d73fff700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d7406a5a0 0x7f4d7414e3c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800c820 tx=0x7f4d6800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:26.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d70ff9700 1 -- 172.21.15.79:0/499124373 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d68011800 con 0x7f4d7406a5a0 2023-06-23T18:37:26.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d70ff9700 1 -- 172.21.15.79:0/499124373 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d68011960 con 0x7f4d7406a5a0 2023-06-23T18:37:26.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d7a88f700 1 -- 172.21.15.79:0/499124373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7414d360 con 0x7f4d7406a5a0 2023-06-23T18:37:26.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d70ff9700 1 -- 172.21.15.79:0/499124373 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d68011d10 con 0x7f4d7406a5a0 2023-06-23T18:37:26.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.506+0000 7f4d7a88f700 1 -- 172.21.15.79:0/499124373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7414d8e0 con 0x7f4d7406a5a0 2023-06-23T18:37:26.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.507+0000 7f4d7a88f700 1 -- 172.21.15.79:0/499124373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d740c39b0 con 0x7f4d7406a5a0 2023-06-23T18:37:26.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.508+0000 7f4d70ff9700 1 -- 172.21.15.79:0/499124373 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d68018360 con 0x7f4d7406a5a0 2023-06-23T18:37:26.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.508+0000 7f4d70ff9700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4d6006ace0 0x7f4d6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.508+0000 7f4d70ff9700 1 -- 172.21.15.79:0/499124373 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4d68089150 con 0x7f4d7406a5a0 2023-06-23T18:37:26.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.508+0000 7f4d737fe700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4d6006ace0 0x7f4d6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.508+0000 7f4d737fe700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4d6006ace0 0x7f4d6006d190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4d740b0990 tx=0x7f4d64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:26.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.512+0000 7f4d70ff9700 1 -- 172.21.15.79:0/499124373 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4d68057fa0 con 0x7f4d7406a5a0 2023-06-23T18:37:26.703 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.698+0000 7f4d7a88f700 1 -- 172.21.15.79:0/499124373 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d74158c00 con 0x7f4d6006ace0 2023-06-23T18:37:26.703 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.698+0000 7f4d70ff9700 1 -- 172.21.15.79:0/499124373 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4d64001cc0 con 0x7f4d6006ace0 2023-06-23T18:37:26.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.700+0000 7f4d7a88f700 1 -- 172.21.15.79:0/499124373 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4d6006ace0 msgr2=0x7f4d6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.700+0000 7f4d7a88f700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4d6006ace0 0x7f4d6006d190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4d740b0990 tx=0x7f4d64009040 comp rx=0 tx=0).stop 2023-06-23T18:37:26.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.700+0000 7f4d7a88f700 1 -- 172.21.15.79:0/499124373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d7406a5a0 msgr2=0x7f4d7414e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.700+0000 7f4d7a88f700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d7406a5a0 0x7f4d7414e3c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800c820 tx=0x7f4d6800cbe0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.700+0000 7f4d7a88f700 1 -- 172.21.15.79:0/499124373 shutdown_connections 2023-06-23T18:37:26.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.700+0000 7f4d7a88f700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d74158830 0x7f4d7414ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.700+0000 7f4d7a88f700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d7406a5a0 0x7f4d7414e3c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.700+0000 7f4d7a88f700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4d6006ace0 0x7f4d6006d190 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.700+0000 7f4d7a88f700 1 --2- 172.21.15.79:0/499124373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d740c5280 0x7f4d74145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.700+0000 7f4d7a88f700 1 -- 172.21.15.79:0/499124373 >> 172.21.15.79:0/499124373 conn(0x7f4d740cf3d0 msgr2=0x7f4d74155e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:26.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.700+0000 7f4d7a88f700 1 -- 172.21.15.79:0/499124373 shutdown_connections 2023-06-23T18:37:26.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.700+0000 7f4d7a88f700 1 -- 172.21.15.79:0/499124373 wait complete. 2023-06-23T18:37:26.708 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 2 to deep-scrub 2023-06-23T18:37:26.717 INFO:tasks.ceph:Scrubbing osd.3 2023-06-23T18:37:26.718 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-06-23T18:37:26.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.836+0000 7f4082361700 1 Processor -- start 2023-06-23T18:37:26.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.836+0000 7f4082361700 1 -- start start 2023-06-23T18:37:26.845 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.837+0000 7f4082361700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f407c06a5a0 0x7f407c160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.845 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.837+0000 7f4082361700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407c160f50 con 0x7f407c0cf1f0 2023-06-23T18:37:26.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.837+0000 7f4082361700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407c0cd410 con 0x7f407c0cb870 2023-06-23T18:37:26.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.837+0000 7f4082361700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407c15cac0 con 0x7f407c06a5a0 2023-06-23T18:37:26.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.837+0000 7f407b7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f407c0cf1f0 0x7f407c069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45406/0 (socket says 172.21.15.79:45406) 2023-06-23T18:37:26.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.837+0000 7f407b7fe700 1 -- 172.21.15.79:0/301118721 learned_addr learned my addr 172.21.15.79:0/301118721 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:26.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.837+0000 7f40808fe700 1 --2- 172.21.15.79:0/301118721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f407c06a5a0 0x7f407c160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.837+0000 7f407affd700 1 -- 172.21.15.79:0/301118721 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138594171 0 0) 0x7f406c001110 con 0x7f407c0cf1f0 2023-06-23T18:37:26.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.837+0000 7f407affd700 1 -- 172.21.15.79:0/301118721 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40680035b0 con 0x7f407c0cf1f0 2023-06-23T18:37:26.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 -- 172.21.15.79:0/301118721 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 180619302 0 0) 0x7f40680035b0 con 0x7f407c0cf1f0 2023-06-23T18:37:26.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 -- 172.21.15.79:0/301118721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f407c06a5a0 msgr2=0x7f407c160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:26.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 --2- 172.21.15.79:0/301118721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f407c06a5a0 0x7f407c160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 -- 172.21.15.79:0/301118721 >> v1:172.21.15.155:6789/0 conn(0x7f407c0cb870 legacy=0x7f407c15d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 -- 172.21.15.79:0/301118721 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c15d040 con 0x7f407c0cf1f0 2023-06-23T18:37:26.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 -- 172.21.15.79:0/301118721 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f406c001800 con 0x7f407c0cf1f0 2023-06-23T18:37:26.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 -- 172.21.15.79:0/301118721 >> v1:172.21.15.79:6789/0 conn(0x7f407c0cf1f0 legacy=0x7f407c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 --2- 172.21.15.79:0/301118721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4068003a30 0x7f4068003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 --2- 172.21.15.79:0/301118721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4068004850 0x7f4068004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 --2- 172.21.15.79:0/301118721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f407c06a5a0 0x7f4068005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 -- 172.21.15.79:0/301118721 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c15cac0 con 0x7f4068003a30 2023-06-23T18:37:26.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 -- 172.21.15.79:0/301118721 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406800baf0 con 0x7f4068004850 2023-06-23T18:37:26.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 -- 172.21.15.79:0/301118721 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406800bcb0 con 0x7f407c06a5a0 2023-06-23T18:37:26.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407bfff700 1 --2- 172.21.15.79:0/301118721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4068003a30 0x7f4068003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f40808fe700 1 --2- 172.21.15.79:0/301118721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f407c06a5a0 0x7f4068005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407b7fe700 1 --2- 172.21.15.79:0/301118721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4068004850 0x7f4068004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407bfff700 1 -- 172.21.15.79:0/301118721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f407c06a5a0 msgr2=0x7f4068005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:26.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407bfff700 1 --2- 172.21.15.79:0/301118721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f407c06a5a0 0x7f4068005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407bfff700 1 -- 172.21.15.79:0/301118721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4068004850 msgr2=0x7f4068004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:26.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407bfff700 1 --2- 172.21.15.79:0/301118721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4068004850 0x7f4068004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407bfff700 1 -- 172.21.15.79:0/301118721 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40680035b0 con 0x7f4068003a30 2023-06-23T18:37:26.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f40808fe700 1 --2- 172.21.15.79:0/301118721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f407c06a5a0 0x7f4068005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:26.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407b7fe700 1 --2- 172.21.15.79:0/301118721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4068004850 0x7f4068004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:26.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407bfff700 1 --2- 172.21.15.79:0/301118721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4068003a30 0x7f4068003e40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4064007fd0 tx=0x7f4068005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d16b838290273a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:26.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 -- 172.21.15.79:0/301118721 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40640012f0 con 0x7f4068003a30 2023-06-23T18:37:26.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 -- 172.21.15.79:0/301118721 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40640052f0 con 0x7f4068003a30 2023-06-23T18:37:26.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.838+0000 7f407affd700 1 -- 172.21.15.79:0/301118721 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4064006860 con 0x7f4068003a30 2023-06-23T18:37:26.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.839+0000 7f4082361700 1 -- 172.21.15.79:0/301118721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4068003a30 msgr2=0x7f4068003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.839+0000 7f4082361700 1 --2- 172.21.15.79:0/301118721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4068003a30 0x7f4068003e40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4064007fd0 tx=0x7f4068005880 comp rx=0 tx=0).stop 2023-06-23T18:37:26.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.839+0000 7f40808fe700 1 -- 172.21.15.79:0/301118721 reap_dead start 2023-06-23T18:37:26.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.839+0000 7f4082361700 1 -- 172.21.15.79:0/301118721 shutdown_connections 2023-06-23T18:37:26.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.839+0000 7f4082361700 1 -- 172.21.15.79:0/301118721 >> 172.21.15.79:0/301118721 conn(0x7f407c06e200 msgr2=0x7f407c0cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:26.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.839+0000 7f4082361700 1 -- 172.21.15.79:0/301118721 shutdown_connections 2023-06-23T18:37:26.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.839+0000 7f4082361700 1 -- 172.21.15.79:0/301118721 wait complete. 2023-06-23T18:37:26.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.840+0000 7f4082361700 1 Processor -- start 2023-06-23T18:37:26.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.840+0000 7f4082361700 1 -- start start 2023-06-23T18:37:26.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f4082361700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4068003a30 0x7f407c15dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f4082361700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f407c0adf00 0x7f407c15f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f4082361700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f407c0ae370 0x7f407c15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f4082361700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c0b5630 con 0x7f407c0ae370 2023-06-23T18:37:26.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f4082361700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c0acb80 con 0x7f407c0adf00 2023-06-23T18:37:26.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f4082361700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407c0ace60 con 0x7f4068003a30 2023-06-23T18:37:26.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f40808fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f407c0ae370 0x7f407c15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f40808fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f407c0ae370 0x7f407c15ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42380/0 (socket says 172.21.15.79:42380) 2023-06-23T18:37:26.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f40808fe700 1 -- 172.21.15.79:0/3054011029 learned_addr learned my addr 172.21.15.79:0/3054011029 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:26.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f407bfff700 1 --2- 172.21.15.79:0/3054011029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4068003a30 0x7f407c15dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f407b7fe700 1 --2- 172.21.15.79:0/3054011029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f407c0adf00 0x7f407c15f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f40808fe700 1 -- 172.21.15.79:0/3054011029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4068003a30 msgr2=0x7f407c15dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f40808fe700 1 --2- 172.21.15.79:0/3054011029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4068003a30 0x7f407c15dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f40808fe700 1 -- 172.21.15.79:0/3054011029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f407c0adf00 msgr2=0x7f407c15f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.841+0000 7f40808fe700 1 --2- 172.21.15.79:0/3054011029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f407c0adf00 0x7f407c15f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.842+0000 7f40808fe700 1 -- 172.21.15.79:0/3054011029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c0ad140 con 0x7f407c0ae370 2023-06-23T18:37:26.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.842+0000 7f40808fe700 1 --2- 172.21.15.79:0/3054011029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f407c0ae370 0x7f407c15ff20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f407c0cf5c0 tx=0x7f40740104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:26.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.842+0000 7f4078ff9700 1 -- 172.21.15.79:0/3054011029 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407401b070 con 0x7f407c0ae370 2023-06-23T18:37:26.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.842+0000 7f4078ff9700 1 -- 172.21.15.79:0/3054011029 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407400f040 con 0x7f407c0ae370 2023-06-23T18:37:26.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.842+0000 7f4082361700 1 -- 172.21.15.79:0/3054011029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407c0ad420 con 0x7f407c0ae370 2023-06-23T18:37:26.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.842+0000 7f4078ff9700 1 -- 172.21.15.79:0/3054011029 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407400e050 con 0x7f407c0ae370 2023-06-23T18:37:26.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.842+0000 7f4082361700 1 -- 172.21.15.79:0/3054011029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407c0ad9e0 con 0x7f407c0ae370 2023-06-23T18:37:26.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.843+0000 7f4082361700 1 -- 172.21.15.79:0/3054011029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f407c0cd660 con 0x7f407c0ae370 2023-06-23T18:37:26.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.843+0000 7f4078ff9700 1 -- 172.21.15.79:0/3054011029 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4074010d80 con 0x7f407c0ae370 2023-06-23T18:37:26.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.843+0000 7f4078ff9700 1 --2- 172.21.15.79:0/3054011029 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f405c06ace0 0x7f405c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.844+0000 7f4078ff9700 1 -- 172.21.15.79:0/3054011029 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4074089290 con 0x7f407c0ae370 2023-06-23T18:37:26.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.844+0000 7f407bfff700 1 --2- 172.21.15.79:0/3054011029 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f405c06ace0 0x7f405c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.844+0000 7f4078ff9700 1 --2- 172.21.15.79:0/3054011029 >> [v2:172.21.15.79:6824/1549244775,v1:172.21.15.79:6825/1549244775] conn(0x7f405c071150 0x7f405c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.844+0000 7f4078ff9700 1 -- 172.21.15.79:0/3054011029 --> [v2:172.21.15.79:6824/1549244775,v1:172.21.15.79:6825/1549244775] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f405c073c30 con 0x7f405c071150 2023-06-23T18:37:26.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.844+0000 7f4078ff9700 1 -- 172.21.15.79:0/3054011029 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f407408b930 con 0x7f407c0ae370 2023-06-23T18:37:26.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.844+0000 7f407b7fe700 1 --2- 172.21.15.79:0/3054011029 >> [v2:172.21.15.79:6824/1549244775,v1:172.21.15.79:6825/1549244775] conn(0x7f405c071150 0x7f405c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.844+0000 7f407bfff700 1 --2- 172.21.15.79:0/3054011029 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f405c06ace0 0x7f405c06d190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4064014040 tx=0x7f4064011000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:26.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.844+0000 7f407b7fe700 1 --2- 172.21.15.79:0/3054011029 >> [v2:172.21.15.79:6824/1549244775,v1:172.21.15.79:6825/1549244775] conn(0x7f405c071150 0x7f405c073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:26.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.845+0000 7f4078ff9700 1 -- 172.21.15.79:0/3054011029 <== osd.3 v2:172.21.15.79:6824/1549244775 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f405c073c30 con 0x7f405c071150 2023-06-23T18:37:26.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.858+0000 7f4082361700 1 -- 172.21.15.79:0/3054011029 --> [v2:172.21.15.79:6824/1549244775,v1:172.21.15.79:6825/1549244775] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f407c0cdc20 con 0x7f405c071150 2023-06-23T18:37:26.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.859+0000 7f4078ff9700 1 -- 172.21.15.79:0/3054011029 <== osd.3 v2:172.21.15.79:6824/1549244775 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f406c00d040 con 0x7f405c071150 2023-06-23T18:37:26.865 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-23T18:37:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-23T18:37:26.866 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-23T18:37:26.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.859+0000 7f4082361700 1 -- 172.21.15.79:0/3054011029 >> [v2:172.21.15.79:6824/1549244775,v1:172.21.15.79:6825/1549244775] conn(0x7f405c071150 msgr2=0x7f405c073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.859+0000 7f4082361700 1 --2- 172.21.15.79:0/3054011029 >> [v2:172.21.15.79:6824/1549244775,v1:172.21.15.79:6825/1549244775] conn(0x7f405c071150 0x7f405c073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.859+0000 7f4082361700 1 -- 172.21.15.79:0/3054011029 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f405c06ace0 msgr2=0x7f405c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.859+0000 7f4082361700 1 --2- 172.21.15.79:0/3054011029 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f405c06ace0 0x7f405c06d190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4064014040 tx=0x7f4064011000 comp rx=0 tx=0).stop 2023-06-23T18:37:26.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.859+0000 7f4082361700 1 -- 172.21.15.79:0/3054011029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f407c0ae370 msgr2=0x7f407c15ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:26.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.859+0000 7f4082361700 1 --2- 172.21.15.79:0/3054011029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f407c0ae370 0x7f407c15ff20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f407c0cf5c0 tx=0x7f40740104a0 comp rx=0 tx=0).stop 2023-06-23T18:37:26.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.859+0000 7f40808fe700 1 -- 172.21.15.79:0/3054011029 reap_dead start 2023-06-23T18:37:26.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.859+0000 7f4082361700 1 -- 172.21.15.79:0/3054011029 shutdown_connections 2023-06-23T18:37:26.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.859+0000 7f4082361700 1 -- 172.21.15.79:0/3054011029 >> 172.21.15.79:0/3054011029 conn(0x7f407c06e200 msgr2=0x7f407c0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:26.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.859+0000 7f4082361700 1 -- 172.21.15.79:0/3054011029 shutdown_connections 2023-06-23T18:37:26.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.859+0000 7f4082361700 1 -- 172.21.15.79:0/3054011029 wait complete. 2023-06-23T18:37:26.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-06-23T18:37:26.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.991+0000 7fa6493d0700 1 Processor -- start 2023-06-23T18:37:26.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.991+0000 7fa6493d0700 1 -- start start 2023-06-23T18:37:26.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.991+0000 7fa6493d0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6440b83c0 0x7fa6440cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.991+0000 7fa6493d0700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa6440cfc60 0x7fa6440d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:26.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.991+0000 7fa6493d0700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6440d0a80 con 0x7fa6440d3710 2023-06-23T18:37:26.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.991+0000 7fa6493d0700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6440b8790 con 0x7fa6440cfc60 2023-06-23T18:37:26.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.991+0000 7fa6493d0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440b4e70 con 0x7fa6440b83c0 2023-06-23T18:37:26.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa6427fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa6440d3710 0x7fa6440cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45428/0 (socket says 172.21.15.79:45428) 2023-06-23T18:37:26.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa6427fc700 1 -- 172.21.15.79:0/149238655 learned_addr learned my addr 172.21.15.79:0/149238655 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:26.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa6437fe700 1 --2- 172.21.15.79:0/149238655 >> v2:172.21.15.155:3300/0 conn(0x7fa6440cfc60 0x7fa6440d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:26.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa642ffd700 1 --2- 172.21.15.79:0/149238655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6440b83c0 0x7fa6440cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa641ffb700 1 -- 172.21.15.79:0/149238655 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862703310 0 0) 0x7fa6440d0a80 con 0x7fa6440d3710 2023-06-23T18:37:27.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa641ffb700 1 -- 172.21.15.79:0/149238655 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa62c0035b0 con 0x7fa6440d3710 2023-06-23T18:37:27.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa641ffb700 1 -- 172.21.15.79:0/149238655 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3502773864 0 0) 0x7fa62c0035b0 con 0x7fa6440d3710 2023-06-23T18:37:27.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa642ffd700 1 -- 172.21.15.79:0/149238655 >> v2:172.21.15.155:3300/0 conn(0x7fa6440cfc60 msgr2=0x7fa6440d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:27.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa642ffd700 1 --2- 172.21.15.79:0/149238655 >> v2:172.21.15.155:3300/0 conn(0x7fa6440cfc60 0x7fa6440d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa642ffd700 1 -- 172.21.15.79:0/149238655 >> v1:172.21.15.79:6789/0 conn(0x7fa6440d3710 legacy=0x7fa6440cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa642ffd700 1 -- 172.21.15.79:0/149238655 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6440b5150 con 0x7fa6440b83c0 2023-06-23T18:37:27.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa642ffd700 1 --2- 172.21.15.79:0/149238655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6440b83c0 0x7fa6440cd200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa638009840 tx=0x7fa638009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=accb0ca0da2ac7f9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa641ffb700 1 -- 172.21.15.79:0/149238655 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634001670 con 0x7fa6440b83c0 2023-06-23T18:37:27.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa641ffb700 1 -- 172.21.15.79:0/149238655 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa638004030 con 0x7fa6440b83c0 2023-06-23T18:37:27.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.992+0000 7fa641ffb700 1 -- 172.21.15.79:0/149238655 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6380043c0 con 0x7fa6440b83c0 2023-06-23T18:37:27.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.993+0000 7fa6493d0700 1 -- 172.21.15.79:0/149238655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6440b83c0 msgr2=0x7fa6440cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.993+0000 7fa6493d0700 1 --2- 172.21.15.79:0/149238655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6440b83c0 0x7fa6440cd200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa638009840 tx=0x7fa638009b50 comp rx=0 tx=0).stop 2023-06-23T18:37:27.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.993+0000 7fa6493d0700 1 -- 172.21.15.79:0/149238655 shutdown_connections 2023-06-23T18:37:27.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.993+0000 7fa6493d0700 1 --2- 172.21.15.79:0/149238655 >> v2:172.21.15.155:3300/0 conn(0x7fa6440cfc60 0x7fa6440d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.993+0000 7fa6493d0700 1 --2- 172.21.15.79:0/149238655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6440b83c0 0x7fa6440cd200 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.993+0000 7fa6493d0700 1 -- 172.21.15.79:0/149238655 >> 172.21.15.79:0/149238655 conn(0x7fa64406b350 msgr2=0x7fa64406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:27.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.993+0000 7fa6493d0700 1 -- 172.21.15.79:0/149238655 shutdown_connections 2023-06-23T18:37:27.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.993+0000 7fa6493d0700 1 -- 172.21.15.79:0/149238655 wait complete. 2023-06-23T18:37:27.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.994+0000 7fa6493d0700 1 Processor -- start 2023-06-23T18:37:27.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.994+0000 7fa6493d0700 1 -- start start 2023-06-23T18:37:27.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa6493d0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6440b83c0 0x7fa644154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa6493d0700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6440cfc60 0x7fa6440c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa6493d0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6440d3710 0x7fa6440d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa6493d0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440c7760 con 0x7fa6440b83c0 2023-06-23T18:37:27.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa6493d0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6441546c0 con 0x7fa6440d3710 2023-06-23T18:37:27.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa6493d0700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440c7b80 con 0x7fa6440cfc60 2023-06-23T18:37:27.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa642ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6440b83c0 0x7fa644154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa642ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6440b83c0 0x7fa644154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42392/0 (socket says 172.21.15.79:42392) 2023-06-23T18:37:27.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa642ffd700 1 -- 172.21.15.79:0/894972186 learned_addr learned my addr 172.21.15.79:0/894972186 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:27.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa6427fc700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6440cfc60 0x7fa6440c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa6437fe700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6440d3710 0x7fa6440d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa642ffd700 1 -- 172.21.15.79:0/894972186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6440cfc60 msgr2=0x7fa6440c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa642ffd700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6440cfc60 0x7fa6440c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa642ffd700 1 -- 172.21.15.79:0/894972186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6440d3710 msgr2=0x7fa6440d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa642ffd700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6440d3710 0x7fa6440d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.995+0000 7fa642ffd700 1 -- 172.21.15.79:0/894972186 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6380094a0 con 0x7fa6440b83c0 2023-06-23T18:37:27.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.996+0000 7fa6437fe700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6440d3710 0x7fa6440d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:27.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.996+0000 7fa642ffd700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6440b83c0 0x7fa644154180 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa63800a010 tx=0x7fa638004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.996+0000 7fa62b7fe700 1 -- 172.21.15.79:0/894972186 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638003c40 con 0x7fa6440b83c0 2023-06-23T18:37:27.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.996+0000 7fa6493d0700 1 -- 172.21.15.79:0/894972186 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6440c8a30 con 0x7fa6440b83c0 2023-06-23T18:37:27.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.996+0000 7fa62b7fe700 1 -- 172.21.15.79:0/894972186 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63801f070 con 0x7fa6440b83c0 2023-06-23T18:37:27.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.996+0000 7fa62b7fe700 1 -- 172.21.15.79:0/894972186 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638011630 con 0x7fa6440b83c0 2023-06-23T18:37:27.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.996+0000 7fa6493d0700 1 -- 172.21.15.79:0/894972186 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6440c6dc0 con 0x7fa6440b83c0 2023-06-23T18:37:27.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.997+0000 7fa6493d0700 1 -- 172.21.15.79:0/894972186 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6440d0fb0 con 0x7fa6440b83c0 2023-06-23T18:37:27.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.997+0000 7fa62b7fe700 1 -- 172.21.15.79:0/894972186 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa638021030 con 0x7fa6440b83c0 2023-06-23T18:37:27.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.998+0000 7fa62b7fe700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa63006ace0 0x7fa63006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.998+0000 7fa62b7fe700 1 -- 172.21.15.79:0/894972186 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa63808a510 con 0x7fa6440b83c0 2023-06-23T18:37:27.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.998+0000 7fa6427fc700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa63006ace0 0x7fa63006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:26.998+0000 7fa6427fc700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa63006ace0 0x7fa63006d190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa634001eb0 tx=0x7fa634001ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.002+0000 7fa62b7fe700 1 -- 172.21.15.79:0/894972186 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa6380558d0 con 0x7fa6440b83c0 2023-06-23T18:37:27.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.190+0000 7fa6493d0700 1 -- 172.21.15.79:0/894972186 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6440d0710 con 0x7fa63006ace0 2023-06-23T18:37:27.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.190+0000 7fa62b7fe700 1 -- 172.21.15.79:0/894972186 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa63400e620 con 0x7fa63006ace0 2023-06-23T18:37:27.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.192+0000 7fa6493d0700 1 -- 172.21.15.79:0/894972186 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa63006ace0 msgr2=0x7fa63006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.192+0000 7fa6493d0700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa63006ace0 0x7fa63006d190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa634001eb0 tx=0x7fa634001ce0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.192+0000 7fa6493d0700 1 -- 172.21.15.79:0/894972186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6440b83c0 msgr2=0x7fa644154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.192+0000 7fa6493d0700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6440b83c0 0x7fa644154180 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa63800a010 tx=0x7fa638004750 comp rx=0 tx=0).stop 2023-06-23T18:37:27.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.192+0000 7fa6493d0700 1 -- 172.21.15.79:0/894972186 shutdown_connections 2023-06-23T18:37:27.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.192+0000 7fa6493d0700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7fa63006ace0 0x7fa63006d190 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.192+0000 7fa6493d0700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6440d3710 0x7fa6440d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.192+0000 7fa6493d0700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6440b83c0 0x7fa644154180 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.192+0000 7fa6493d0700 1 --2- 172.21.15.79:0/894972186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6440cfc60 0x7fa6440c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.192+0000 7fa6493d0700 1 -- 172.21.15.79:0/894972186 >> 172.21.15.79:0/894972186 conn(0x7fa64406b350 msgr2=0x7fa6440b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:27.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.192+0000 7fa6493d0700 1 -- 172.21.15.79:0/894972186 shutdown_connections 2023-06-23T18:37:27.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.192+0000 7fa6493d0700 1 -- 172.21.15.79:0/894972186 wait complete. 2023-06-23T18:37:27.200 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 3 to deep-scrub 2023-06-23T18:37:27.209 INFO:tasks.ceph:Scrubbing osd.4 2023-06-23T18:37:27.209 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-06-23T18:37:27.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.331+0000 7f6ee2acc700 1 Processor -- start 2023-06-23T18:37:27.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.331+0000 7f6ee2acc700 1 -- start start 2023-06-23T18:37:27.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6ee2acc700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6edc15a2d0 0x7f6edc06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6ee2acc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6edc071ba0 0x7f6edc15e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6ee2acc700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6edc15a6a0 con 0x7f6edc0aee00 2023-06-23T18:37:27.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6ee2acc700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6edc0cc170 con 0x7f6edc15a2d0 2023-06-23T18:37:27.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6ee2acc700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0cb910 con 0x7f6edc071ba0 2023-06-23T18:37:27.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edbfff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f6edc0aee00 0x7f6edc071490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45444/0 (socket says 172.21.15.79:45444) 2023-06-23T18:37:27.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edbfff700 1 -- 172.21.15.79:0/4196029386 learned_addr learned my addr 172.21.15.79:0/4196029386 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:27.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6ee0868700 1 --2- 172.21.15.79:0/4196029386 >> v2:172.21.15.155:3300/0 conn(0x7f6edc15a2d0 0x7f6edc06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6ee1069700 1 --2- 172.21.15.79:0/4196029386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6edc071ba0 0x7f6edc15e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 -- 172.21.15.79:0/4196029386 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552805640 0 0) 0x7f6ec8001110 con 0x7f6edc0aee00 2023-06-23T18:37:27.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 -- 172.21.15.79:0/4196029386 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ecc0035b0 con 0x7f6edc0aee00 2023-06-23T18:37:27.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 -- 172.21.15.79:0/4196029386 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2546912657 0 0) 0x7f6ecc0035b0 con 0x7f6edc0aee00 2023-06-23T18:37:27.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 -- 172.21.15.79:0/4196029386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6edc071ba0 msgr2=0x7f6edc15e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:27.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 --2- 172.21.15.79:0/4196029386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6edc071ba0 0x7f6edc15e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 -- 172.21.15.79:0/4196029386 >> v2:172.21.15.155:3300/0 conn(0x7f6edc15a2d0 msgr2=0x7f6edc06eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:27.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 --2- 172.21.15.79:0/4196029386 >> v2:172.21.15.155:3300/0 conn(0x7f6edc15a2d0 0x7f6edc06eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 -- 172.21.15.79:0/4196029386 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc0ca200 con 0x7f6edc0aee00 2023-06-23T18:37:27.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 -- 172.21.15.79:0/4196029386 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f6ec8001800 con 0x7f6edc0aee00 2023-06-23T18:37:27.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 -- 172.21.15.79:0/4196029386 >> v1:172.21.15.79:6789/0 conn(0x7f6edc0aee00 legacy=0x7f6edc071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 --2- 172.21.15.79:0/4196029386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ecc003a00 0x7f6ecc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 --2- 172.21.15.79:0/4196029386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6edc071ba0 0x7f6ecc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 --2- 172.21.15.79:0/4196029386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ecc004d90 0x7f6ecc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 -- 172.21.15.79:0/4196029386 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0cb910 con 0x7f6ecc004d90 2023-06-23T18:37:27.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 -- 172.21.15.79:0/4196029386 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc00ba10 con 0x7f6ecc003a00 2023-06-23T18:37:27.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.332+0000 7f6edb7fe700 1 -- 172.21.15.79:0/4196029386 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc00bb50 con 0x7f6edc071ba0 2023-06-23T18:37:27.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.333+0000 7f6ee1069700 1 --2- 172.21.15.79:0/4196029386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ecc004d90 0x7f6ecc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.333+0000 7f6edbfff700 1 --2- 172.21.15.79:0/4196029386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6edc071ba0 0x7f6ecc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.333+0000 7f6ee0868700 1 --2- 172.21.15.79:0/4196029386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ecc003a00 0x7f6ecc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.333+0000 7f6ee1069700 1 -- 172.21.15.79:0/4196029386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6edc071ba0 msgr2=0x7f6ecc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:27.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.333+0000 7f6ee1069700 1 --2- 172.21.15.79:0/4196029386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6edc071ba0 0x7f6ecc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.333+0000 7f6ee1069700 1 -- 172.21.15.79:0/4196029386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ecc003a00 msgr2=0x7f6ecc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:27.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.333+0000 7f6ee1069700 1 --2- 172.21.15.79:0/4196029386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ecc003a00 0x7f6ecc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.333+0000 7f6ee1069700 1 -- 172.21.15.79:0/4196029386 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc0035b0 con 0x7f6ecc004d90 2023-06-23T18:37:27.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.333+0000 7f6ee1069700 1 --2- 172.21.15.79:0/4196029386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ecc004d90 0x7f6ecc005200 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0726d0 tx=0x7f6ecc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc1c652d6823c68 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.333+0000 7f6edb7fe700 1 -- 172.21.15.79:0/4196029386 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400ebd0 con 0x7f6ecc004d90 2023-06-23T18:37:27.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.333+0000 7f6edb7fe700 1 -- 172.21.15.79:0/4196029386 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed400f040 con 0x7f6ecc004d90 2023-06-23T18:37:27.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.333+0000 7f6edb7fe700 1 -- 172.21.15.79:0/4196029386 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4014480 con 0x7f6ecc004d90 2023-06-23T18:37:27.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.334+0000 7f6ee2acc700 1 -- 172.21.15.79:0/4196029386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ecc004d90 msgr2=0x7f6ecc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.334+0000 7f6ee2acc700 1 --2- 172.21.15.79:0/4196029386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ecc004d90 0x7f6ecc005200 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0726d0 tx=0x7f6ecc005880 comp rx=0 tx=0).stop 2023-06-23T18:37:27.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.334+0000 7f6ee1069700 1 -- 172.21.15.79:0/4196029386 reap_dead start 2023-06-23T18:37:27.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.334+0000 7f6ee2acc700 1 -- 172.21.15.79:0/4196029386 shutdown_connections 2023-06-23T18:37:27.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.334+0000 7f6ee2acc700 1 -- 172.21.15.79:0/4196029386 >> 172.21.15.79:0/4196029386 conn(0x7f6edc06e200 msgr2=0x7f6edc0cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:27.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.334+0000 7f6ee2acc700 1 -- 172.21.15.79:0/4196029386 shutdown_connections 2023-06-23T18:37:27.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.334+0000 7f6ee2acc700 1 -- 172.21.15.79:0/4196029386 wait complete. 2023-06-23T18:37:27.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.335+0000 7f6ee2acc700 1 Processor -- start 2023-06-23T18:37:27.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.335+0000 7f6ee2acc700 1 -- start start 2023-06-23T18:37:27.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee2acc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6edc15aa50 0x7f6edc0c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee2acc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6edc15d460 0x7f6edc15ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee2acc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6edc15b360 0x7f6edc0c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee2acc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0c55c0 con 0x7f6edc15b360 2023-06-23T18:37:27.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee2acc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0c58a0 con 0x7f6edc15aa50 2023-06-23T18:37:27.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee2acc700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0c5b80 con 0x7f6edc15d460 2023-06-23T18:37:27.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee1069700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6edc15b360 0x7f6edc0c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee1069700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6edc15b360 0x7f6edc0c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42414/0 (socket says 172.21.15.79:42414) 2023-06-23T18:37:27.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee1069700 1 -- 172.21.15.79:0/2617913639 learned_addr learned my addr 172.21.15.79:0/2617913639 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:27.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee1069700 1 -- 172.21.15.79:0/2617913639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6edc15d460 msgr2=0x7f6edc15ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee1069700 1 --2- 172.21.15.79:0/2617913639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6edc15d460 0x7f6edc15ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee1069700 1 -- 172.21.15.79:0/2617913639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6edc15aa50 msgr2=0x7f6edc0c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee0868700 1 --2- 172.21.15.79:0/2617913639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6edc15aa50 0x7f6edc0c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee1069700 1 --2- 172.21.15.79:0/2617913639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6edc15aa50 0x7f6edc0c4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee1069700 1 -- 172.21.15.79:0/2617913639 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc0c7ce0 con 0x7f6edc15b360 2023-06-23T18:37:27.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ee1069700 1 --2- 172.21.15.79:0/2617913639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6edc15b360 0x7f6edc0c84e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009f60 tx=0x7f6ed4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ed97fa700 1 -- 172.21.15.79:0/2617913639 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4007000 con 0x7f6edc15b360 2023-06-23T18:37:27.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.336+0000 7f6ed97fa700 1 -- 172.21.15.79:0/2617913639 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed4003e70 con 0x7f6edc15b360 2023-06-23T18:37:27.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.337+0000 7f6ee2acc700 1 -- 172.21.15.79:0/2617913639 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6edc0c7fc0 con 0x7f6edc15b360 2023-06-23T18:37:27.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.337+0000 7f6ed97fa700 1 -- 172.21.15.79:0/2617913639 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4014ab0 con 0x7f6edc15b360 2023-06-23T18:37:27.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.337+0000 7f6ee2acc700 1 -- 172.21.15.79:0/2617913639 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6edc0c5290 con 0x7f6edc15b360 2023-06-23T18:37:27.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.337+0000 7f6ee2acc700 1 -- 172.21.15.79:0/2617913639 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6edc0cbe90 con 0x7f6edc15b360 2023-06-23T18:37:27.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.338+0000 7f6ed97fa700 1 -- 172.21.15.79:0/2617913639 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6ed4014c10 con 0x7f6edc15b360 2023-06-23T18:37:27.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.338+0000 7f6ed97fa700 1 --2- 172.21.15.79:0/2617913639 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f6ec006ace0 0x7f6ec006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.338+0000 7f6ed97fa700 1 -- 172.21.15.79:0/2617913639 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6ed408acd0 con 0x7f6edc15b360 2023-06-23T18:37:27.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.338+0000 7f6ee0868700 1 --2- 172.21.15.79:0/2617913639 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f6ec006ace0 0x7f6ec006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.338+0000 7f6ed97fa700 1 --2- 172.21.15.79:0/2617913639 >> [v2:172.21.15.155:6816/2160815572,v1:172.21.15.155:6817/2160815572] conn(0x7f6ec0071150 0x7f6ec0073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.338+0000 7f6ed97fa700 1 -- 172.21.15.79:0/2617913639 --> [v2:172.21.15.155:6816/2160815572,v1:172.21.15.155:6817/2160815572] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ec0073c30 con 0x7f6ec0071150 2023-06-23T18:37:27.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.338+0000 7f6ed97fa700 1 -- 172.21.15.79:0/2617913639 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ed408d370 con 0x7f6edc15b360 2023-06-23T18:37:27.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.339+0000 7f6ee0868700 1 --2- 172.21.15.79:0/2617913639 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f6ec006ace0 0x7f6ec006d190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0009aa0 tx=0x7f6ed0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.339+0000 7f6edbfff700 1 --2- 172.21.15.79:0/2617913639 >> [v2:172.21.15.155:6816/2160815572,v1:172.21.15.155:6817/2160815572] conn(0x7f6ec0071150 0x7f6ec0073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.339+0000 7f6edbfff700 1 --2- 172.21.15.79:0/2617913639 >> [v2:172.21.15.155:6816/2160815572,v1:172.21.15.155:6817/2160815572] conn(0x7f6ec0071150 0x7f6ec0073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.340+0000 7f6ed97fa700 1 -- 172.21.15.79:0/2617913639 <== osd.4 v2:172.21.15.155:6816/2160815572 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6ec0073c30 con 0x7f6ec0071150 2023-06-23T18:37:27.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.354+0000 7f6ee2acc700 1 -- 172.21.15.79:0/2617913639 --> [v2:172.21.15.155:6816/2160815572,v1:172.21.15.155:6817/2160815572] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6edc0cbbb0 con 0x7f6ec0071150 2023-06-23T18:37:27.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.355+0000 7f6ed97fa700 1 -- 172.21.15.79:0/2617913639 <== osd.4 v2:172.21.15.155:6816/2160815572 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6ec8001d10 con 0x7f6ec0071150 2023-06-23T18:37:27.360 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-23T18:37:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-23T18:37:27.360 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-23T18:37:27.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.355+0000 7f6ee2acc700 1 -- 172.21.15.79:0/2617913639 >> [v2:172.21.15.155:6816/2160815572,v1:172.21.15.155:6817/2160815572] conn(0x7f6ec0071150 msgr2=0x7f6ec0073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.355+0000 7f6ee2acc700 1 --2- 172.21.15.79:0/2617913639 >> [v2:172.21.15.155:6816/2160815572,v1:172.21.15.155:6817/2160815572] conn(0x7f6ec0071150 0x7f6ec0073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.355+0000 7f6ee2acc700 1 -- 172.21.15.79:0/2617913639 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f6ec006ace0 msgr2=0x7f6ec006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.355+0000 7f6ee2acc700 1 --2- 172.21.15.79:0/2617913639 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f6ec006ace0 0x7f6ec006d190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0009aa0 tx=0x7f6ed0008040 comp rx=0 tx=0).stop 2023-06-23T18:37:27.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.355+0000 7f6ee2acc700 1 -- 172.21.15.79:0/2617913639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6edc15b360 msgr2=0x7f6edc0c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.355+0000 7f6ee2acc700 1 --2- 172.21.15.79:0/2617913639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6edc15b360 0x7f6edc0c84e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009f60 tx=0x7f6ed4009750 comp rx=0 tx=0).stop 2023-06-23T18:37:27.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.355+0000 7f6ee1069700 1 -- 172.21.15.79:0/2617913639 reap_dead start 2023-06-23T18:37:27.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.355+0000 7f6ee2acc700 1 -- 172.21.15.79:0/2617913639 shutdown_connections 2023-06-23T18:37:27.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.355+0000 7f6ee2acc700 1 -- 172.21.15.79:0/2617913639 >> 172.21.15.79:0/2617913639 conn(0x7f6edc06e200 msgr2=0x7f6edc0b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:27.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.355+0000 7f6ee2acc700 1 -- 172.21.15.79:0/2617913639 shutdown_connections 2023-06-23T18:37:27.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.355+0000 7f6ee2acc700 1 -- 172.21.15.79:0/2617913639 wait complete. 2023-06-23T18:37:27.371 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-06-23T18:37:27.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.487+0000 7f71d96c5700 1 Processor -- start 2023-06-23T18:37:27.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.487+0000 7f71d96c5700 1 -- start start 2023-06-23T18:37:27.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d96c5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d415c6a0 0x7f71d406c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d96c5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f71d40b2470 0x7f71d40b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d96c5700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d406b8a0 con 0x7f71d40b3010 2023-06-23T18:37:27.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d96c5700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d406b6b0 con 0x7f71d40b2470 2023-06-23T18:37:27.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d96c5700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d415ca70 con 0x7f71d415c6a0 2023-06-23T18:37:27.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d37fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f71d40b2470 0x7f71d40b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d37fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f71d40b2470 0x7f71d40b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42424/0 (socket says 172.21.15.79:42424) 2023-06-23T18:37:27.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d37fe700 1 -- 172.21.15.79:0/15684304 learned_addr learned my addr 172.21.15.79:0/15684304 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:27.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d2ffd700 1 --2- 172.21.15.79:0/15684304 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d415c6a0 0x7f71d406c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d37fe700 1 -- 172.21.15.79:0/15684304 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d415c6a0 msgr2=0x7f71d406c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:27.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d37fe700 1 --2- 172.21.15.79:0/15684304 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d415c6a0 0x7f71d406c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d37fe700 1 -- 172.21.15.79:0/15684304 >> v1:172.21.15.155:6789/0 conn(0x7f71d40b3010 legacy=0x7f71d406afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d37fe700 1 -- 172.21.15.79:0/15684304 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d406bb80 con 0x7f71d40b2470 2023-06-23T18:37:27.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d37fe700 1 --2- 172.21.15.79:0/15684304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d40b2470 0x7f71d40b28f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009e50 tx=0x7f71cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a8bc257d0e4a9a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d1ffb700 1 -- 172.21.15.79:0/15684304 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc00e070 con 0x7f71d40b2470 2023-06-23T18:37:27.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d1ffb700 1 -- 172.21.15.79:0/15684304 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc00b9c0 con 0x7f71d40b2470 2023-06-23T18:37:27.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.488+0000 7f71d1ffb700 1 -- 172.21.15.79:0/15684304 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc00bd70 con 0x7f71d40b2470 2023-06-23T18:37:27.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.489+0000 7f71d96c5700 1 -- 172.21.15.79:0/15684304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d40b2470 msgr2=0x7f71d40b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.489+0000 7f71d96c5700 1 --2- 172.21.15.79:0/15684304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d40b2470 0x7f71d40b28f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009e50 tx=0x7f71cc00b330 comp rx=0 tx=0).stop 2023-06-23T18:37:27.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.489+0000 7f71d96c5700 1 -- 172.21.15.79:0/15684304 shutdown_connections 2023-06-23T18:37:27.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.489+0000 7f71d96c5700 1 --2- 172.21.15.79:0/15684304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d40b2470 0x7f71d40b28f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.489+0000 7f71d96c5700 1 --2- 172.21.15.79:0/15684304 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d415c6a0 0x7f71d406c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.489+0000 7f71d96c5700 1 -- 172.21.15.79:0/15684304 >> 172.21.15.79:0/15684304 conn(0x7f71d406e640 msgr2=0x7f71d406ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:27.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.489+0000 7f71d96c5700 1 -- 172.21.15.79:0/15684304 shutdown_connections 2023-06-23T18:37:27.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.489+0000 7f71d96c5700 1 -- 172.21.15.79:0/15684304 wait complete. 2023-06-23T18:37:27.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.490+0000 7f71d96c5700 1 Processor -- start 2023-06-23T18:37:27.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.490+0000 7f71d96c5700 1 -- start start 2023-06-23T18:37:27.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d96c5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d40b2470 0x7f71d4146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d96c5700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40b3010 0x7f71d40c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d96c5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d415c6a0 0x7f71d40c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d96c5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d415dca0 con 0x7f71d40b2470 2023-06-23T18:37:27.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d96c5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d4147530 con 0x7f71d415c6a0 2023-06-23T18:37:27.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d96c5700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d4147840 con 0x7f71d40b3010 2023-06-23T18:37:27.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d2ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d40b2470 0x7f71d4146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d2ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d40b2470 0x7f71d4146ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42426/0 (socket says 172.21.15.79:42426) 2023-06-23T18:37:27.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d2ffd700 1 -- 172.21.15.79:0/2678509460 learned_addr learned my addr 172.21.15.79:0/2678509460 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:27.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d37fe700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d415c6a0 0x7f71d40c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d27fc700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40b3010 0x7f71d40c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d2ffd700 1 -- 172.21.15.79:0/2678509460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40b3010 msgr2=0x7f71d40c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d2ffd700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40b3010 0x7f71d40c9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d2ffd700 1 -- 172.21.15.79:0/2678509460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d415c6a0 msgr2=0x7f71d40c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d2ffd700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d415c6a0 0x7f71d40c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d2ffd700 1 -- 172.21.15.79:0/2678509460 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc009b30 con 0x7f71d40b2470 2023-06-23T18:37:27.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d27fc700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40b3010 0x7f71d40c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:27.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.491+0000 7f71d2ffd700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d40b2470 0x7f71d4146ff0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f71c400d800 tx=0x7f71c400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.492+0000 7f71bb7fe700 1 -- 172.21.15.79:0/2678509460 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c40125f0 con 0x7f71d40b2470 2023-06-23T18:37:27.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.492+0000 7f71bb7fe700 1 -- 172.21.15.79:0/2678509460 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c4012750 con 0x7f71d40b2470 2023-06-23T18:37:27.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.492+0000 7f71d96c5700 1 -- 172.21.15.79:0/2678509460 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d40ca460 con 0x7f71d40b2470 2023-06-23T18:37:27.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.492+0000 7f71bb7fe700 1 -- 172.21.15.79:0/2678509460 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4012b00 con 0x7f71d40b2470 2023-06-23T18:37:27.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.492+0000 7f71d96c5700 1 -- 172.21.15.79:0/2678509460 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d415df40 con 0x7f71d40b2470 2023-06-23T18:37:27.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.493+0000 7f71d96c5700 1 -- 172.21.15.79:0/2678509460 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d415e2b0 con 0x7f71d40b2470 2023-06-23T18:37:27.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.493+0000 7f71bb7fe700 1 -- 172.21.15.79:0/2678509460 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f71c4012c60 con 0x7f71d40b2470 2023-06-23T18:37:27.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.493+0000 7f71bb7fe700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f71c006ace0 0x7f71c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.493+0000 7f71bb7fe700 1 -- 172.21.15.79:0/2678509460 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f71c4089540 con 0x7f71d40b2470 2023-06-23T18:37:27.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.493+0000 7f71d27fc700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f71c006ace0 0x7f71c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.494+0000 7f71d27fc700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f71c006ace0 0x7f71c006d190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f71c8000f60 tx=0x7f71c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.498+0000 7f71bb7fe700 1 -- 172.21.15.79:0/2678509460 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f71c4054870 con 0x7f71d40b2470 2023-06-23T18:37:27.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.682+0000 7f71d96c5700 1 -- 172.21.15.79:0/2678509460 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f71d406c140 con 0x7f71c006ace0 2023-06-23T18:37:27.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.682+0000 7f71bb7fe700 1 -- 172.21.15.79:0/2678509460 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f71c80014f0 con 0x7f71c006ace0 2023-06-23T18:37:27.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.684+0000 7f71d96c5700 1 -- 172.21.15.79:0/2678509460 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f71c006ace0 msgr2=0x7f71c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.684+0000 7f71d96c5700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f71c006ace0 0x7f71c006d190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f71c8000f60 tx=0x7f71c8009040 comp rx=0 tx=0).stop 2023-06-23T18:37:27.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.685+0000 7f71d96c5700 1 -- 172.21.15.79:0/2678509460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d40b2470 msgr2=0x7f71d4146ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.685+0000 7f71d96c5700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d40b2470 0x7f71d4146ff0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f71c400d800 tx=0x7f71c400db10 comp rx=0 tx=0).stop 2023-06-23T18:37:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.685+0000 7f71d96c5700 1 -- 172.21.15.79:0/2678509460 shutdown_connections 2023-06-23T18:37:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.685+0000 7f71d96c5700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f71c006ace0 0x7f71c006d190 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.685+0000 7f71d96c5700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d415c6a0 0x7f71d40c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.685+0000 7f71d96c5700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d40b2470 0x7f71d4146ff0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.685+0000 7f71d96c5700 1 --2- 172.21.15.79:0/2678509460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40b3010 0x7f71d40c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.685+0000 7f71d96c5700 1 -- 172.21.15.79:0/2678509460 >> 172.21.15.79:0/2678509460 conn(0x7f71d406e640 msgr2=0x7f71d40d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.685+0000 7f71d96c5700 1 -- 172.21.15.79:0/2678509460 shutdown_connections 2023-06-23T18:37:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.685+0000 7f71d96c5700 1 -- 172.21.15.79:0/2678509460 wait complete. 2023-06-23T18:37:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 4 to deep-scrub 2023-06-23T18:37:27.701 INFO:tasks.ceph:Scrubbing osd.5 2023-06-23T18:37:27.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-06-23T18:37:27.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.821+0000 7f01496db700 1 Processor -- start 2023-06-23T18:37:27.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.821+0000 7f01496db700 1 -- start start 2023-06-23T18:37:27.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.822+0000 7f01496db700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f014415c960 0x7f01440cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.822+0000 7f01496db700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01440cfef0 0x7f014406afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.822+0000 7f01496db700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014415cd30 con 0x7f014406b4e0 2023-06-23T18:37:27.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.822+0000 7f01496db700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014406b980 con 0x7f014415c960 2023-06-23T18:37:27.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.822+0000 7f01496db700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01440d1ad0 con 0x7f01440cfef0 2023-06-23T18:37:27.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.822+0000 7f01427fc700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01440cfef0 0x7f014406afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.822+0000 7f0142ffd700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f014415c960 0x7f01440cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.822+0000 7f01437fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f014406b4e0 0x7f01440b0a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:53748/0 (socket says 172.21.15.79:53748) 2023-06-23T18:37:27.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.822+0000 7f01437fe700 1 -- 172.21.15.79:0/2288586943 learned_addr learned my addr 172.21.15.79:0/2288586943 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:27.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.822+0000 7f0142ffd700 1 -- 172.21.15.79:0/2288586943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01440cfef0 msgr2=0x7f014406afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:27.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.822+0000 7f0142ffd700 1 --2- 172.21.15.79:0/2288586943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01440cfef0 0x7f014406afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.822+0000 7f0142ffd700 1 -- 172.21.15.79:0/2288586943 >> v1:172.21.15.155:6789/0 conn(0x7f014406b4e0 legacy=0x7f01440b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.822+0000 7f0142ffd700 1 -- 172.21.15.79:0/2288586943 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01440d1640 con 0x7f014415c960 2023-06-23T18:37:27.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.823+0000 7f0142ffd700 1 --2- 172.21.15.79:0/2288586943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f014415c960 0x7f01440cdda0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0134009840 tx=0x7f0134009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=482d0d5d6ade0629 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.823+0000 7f0141ffb700 1 -- 172.21.15.79:0/2288586943 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134011830 con 0x7f014415c960 2023-06-23T18:37:27.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.823+0000 7f0141ffb700 1 -- 172.21.15.79:0/2288586943 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0134011990 con 0x7f014415c960 2023-06-23T18:37:27.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.823+0000 7f0141ffb700 1 -- 172.21.15.79:0/2288586943 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134011d20 con 0x7f014415c960 2023-06-23T18:37:27.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.823+0000 7f01496db700 1 -- 172.21.15.79:0/2288586943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f014415c960 msgr2=0x7f01440cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.823+0000 7f01496db700 1 --2- 172.21.15.79:0/2288586943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f014415c960 0x7f01440cdda0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0134009840 tx=0x7f0134009b50 comp rx=0 tx=0).stop 2023-06-23T18:37:27.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.823+0000 7f01496db700 1 -- 172.21.15.79:0/2288586943 shutdown_connections 2023-06-23T18:37:27.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.823+0000 7f01496db700 1 --2- 172.21.15.79:0/2288586943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f014415c960 0x7f01440cdda0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.823+0000 7f01496db700 1 --2- 172.21.15.79:0/2288586943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01440cfef0 0x7f014406afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.823+0000 7f01496db700 1 -- 172.21.15.79:0/2288586943 >> 172.21.15.79:0/2288586943 conn(0x7f014406a7b0 msgr2=0x7f0144099de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:27.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.823+0000 7f01496db700 1 -- 172.21.15.79:0/2288586943 shutdown_connections 2023-06-23T18:37:27.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.823+0000 7f01496db700 1 -- 172.21.15.79:0/2288586943 wait complete. 2023-06-23T18:37:27.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.825+0000 7f01496db700 1 Processor -- start 2023-06-23T18:37:27.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.825+0000 7f01496db700 1 -- start start 2023-06-23T18:37:27.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.825+0000 7f01496db700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f014406b4e0 0x7f01440cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.825+0000 7f01496db700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01440cfef0 0x7f01440cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.825+0000 7f01496db700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f014415c960 0x7f01440b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.825+0000 7f01496db700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01440cc8c0 con 0x7f01440cfef0 2023-06-23T18:37:27.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.825+0000 7f01496db700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01440ccba0 con 0x7f014406b4e0 2023-06-23T18:37:27.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.825+0000 7f01496db700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01440cb2e0 con 0x7f014415c960 2023-06-23T18:37:27.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f01427fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01440cfef0 0x7f01440cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f01427fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01440cfef0 0x7f01440cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42442/0 (socket says 172.21.15.79:42442) 2023-06-23T18:37:27.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f01427fc700 1 -- 172.21.15.79:0/1987819323 learned_addr learned my addr 172.21.15.79:0/1987819323 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:27.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f0142ffd700 1 --2- 172.21.15.79:0/1987819323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f014406b4e0 0x7f01440cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f01437fe700 1 --2- 172.21.15.79:0/1987819323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f014415c960 0x7f01440b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f01427fc700 1 -- 172.21.15.79:0/1987819323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f014415c960 msgr2=0x7f01440b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f01427fc700 1 --2- 172.21.15.79:0/1987819323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f014415c960 0x7f01440b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f01427fc700 1 -- 172.21.15.79:0/1987819323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f014406b4e0 msgr2=0x7f01440cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f01427fc700 1 --2- 172.21.15.79:0/1987819323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f014406b4e0 0x7f01440cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f01427fc700 1 -- 172.21.15.79:0/1987819323 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01340094a0 con 0x7f01440cfef0 2023-06-23T18:37:27.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f01427fc700 1 --2- 172.21.15.79:0/1987819323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01440cfef0 0x7f01440cbae0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f013800e980 tx=0x7f013800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f012b7fe700 1 -- 172.21.15.79:0/1987819323 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138009710 con 0x7f01440cfef0 2023-06-23T18:37:27.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f012b7fe700 1 -- 172.21.15.79:0/1987819323 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013800c690 con 0x7f01440cfef0 2023-06-23T18:37:27.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f01496db700 1 -- 172.21.15.79:0/1987819323 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01440cb5c0 con 0x7f01440cfef0 2023-06-23T18:37:27.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f012b7fe700 1 -- 172.21.15.79:0/1987819323 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013800ca20 con 0x7f01440cfef0 2023-06-23T18:37:27.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.826+0000 7f01496db700 1 -- 172.21.15.79:0/1987819323 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01440cc020 con 0x7f01440cfef0 2023-06-23T18:37:27.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.827+0000 7f01496db700 1 -- 172.21.15.79:0/1987819323 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f01440d0d10 con 0x7f01440cfef0 2023-06-23T18:37:27.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.828+0000 7f012b7fe700 1 -- 172.21.15.79:0/1987819323 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f013800cb80 con 0x7f01440cfef0 2023-06-23T18:37:27.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.828+0000 7f012b7fe700 1 --2- 172.21.15.79:0/1987819323 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f013006ace0 0x7f013006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.828+0000 7f012b7fe700 1 -- 172.21.15.79:0/1987819323 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0138089080 con 0x7f01440cfef0 2023-06-23T18:37:27.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.828+0000 7f0142ffd700 1 --2- 172.21.15.79:0/1987819323 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f013006ace0 0x7f013006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.828+0000 7f012b7fe700 1 --2- 172.21.15.79:0/1987819323 >> [v2:172.21.15.155:6800/4051222998,v1:172.21.15.155:6803/4051222998] conn(0x7f0130071150 0x7f0130073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.828+0000 7f012b7fe700 1 -- 172.21.15.79:0/1987819323 --> [v2:172.21.15.155:6800/4051222998,v1:172.21.15.155:6803/4051222998] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0130073c30 con 0x7f0130071150 2023-06-23T18:37:27.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.828+0000 7f012b7fe700 1 -- 172.21.15.79:0/1987819323 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f013808b720 con 0x7f01440cfef0 2023-06-23T18:37:27.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.828+0000 7f01437fe700 1 --2- 172.21.15.79:0/1987819323 >> [v2:172.21.15.155:6800/4051222998,v1:172.21.15.155:6803/4051222998] conn(0x7f0130071150 0x7f0130073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.828+0000 7f0142ffd700 1 --2- 172.21.15.79:0/1987819323 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f013006ace0 0x7f013006d190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0134000c00 tx=0x7f0134008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.829+0000 7f01437fe700 1 --2- 172.21.15.79:0/1987819323 >> [v2:172.21.15.155:6800/4051222998,v1:172.21.15.155:6803/4051222998] conn(0x7f0130071150 0x7f0130073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.830+0000 7f012b7fe700 1 -- 172.21.15.79:0/1987819323 <== osd.5 v2:172.21.15.155:6800/4051222998 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0130073c30 con 0x7f0130071150 2023-06-23T18:37:27.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.844+0000 7f01496db700 1 -- 172.21.15.79:0/1987819323 --> [v2:172.21.15.155:6800/4051222998,v1:172.21.15.155:6803/4051222998] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f01440d0a30 con 0x7f0130071150 2023-06-23T18:37:27.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.844+0000 7f012b7fe700 1 -- 172.21.15.79:0/1987819323 <== osd.5 v2:172.21.15.155:6800/4051222998 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f013c001e60 con 0x7f0130071150 2023-06-23T18:37:27.849 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-23T18:37:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-23T18:37:27.850 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-23T18:37:27.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.844+0000 7f01496db700 1 -- 172.21.15.79:0/1987819323 >> [v2:172.21.15.155:6800/4051222998,v1:172.21.15.155:6803/4051222998] conn(0x7f0130071150 msgr2=0x7f0130073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.844+0000 7f01496db700 1 --2- 172.21.15.79:0/1987819323 >> [v2:172.21.15.155:6800/4051222998,v1:172.21.15.155:6803/4051222998] conn(0x7f0130071150 0x7f0130073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.844+0000 7f01496db700 1 -- 172.21.15.79:0/1987819323 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f013006ace0 msgr2=0x7f013006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.844+0000 7f01496db700 1 --2- 172.21.15.79:0/1987819323 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f013006ace0 0x7f013006d190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0134000c00 tx=0x7f0134008030 comp rx=0 tx=0).stop 2023-06-23T18:37:27.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.844+0000 7f01496db700 1 -- 172.21.15.79:0/1987819323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01440cfef0 msgr2=0x7f01440cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.844+0000 7f01496db700 1 --2- 172.21.15.79:0/1987819323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01440cfef0 0x7f01440cbae0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f013800e980 tx=0x7f013800ec90 comp rx=0 tx=0).stop 2023-06-23T18:37:27.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.844+0000 7f01437fe700 1 -- 172.21.15.79:0/1987819323 reap_dead start 2023-06-23T18:37:27.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.844+0000 7f01496db700 1 -- 172.21.15.79:0/1987819323 shutdown_connections 2023-06-23T18:37:27.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.844+0000 7f01496db700 1 -- 172.21.15.79:0/1987819323 >> 172.21.15.79:0/1987819323 conn(0x7f014406a7b0 msgr2=0x7f01440aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:27.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.845+0000 7f01496db700 1 -- 172.21.15.79:0/1987819323 shutdown_connections 2023-06-23T18:37:27.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.845+0000 7f01496db700 1 -- 172.21.15.79:0/1987819323 wait complete. 2023-06-23T18:37:27.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-06-23T18:37:27.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.976+0000 7f4d7e968700 1 Processor -- start 2023-06-23T18:37:27.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.976+0000 7f4d7e968700 1 -- start start 2023-06-23T18:37:27.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d7e968700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4d78159170 0x7f4d780cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d7e968700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4d780cd900 0x7f4d780ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d7e968700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d780ce620 0x7f4d78071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d7e968700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d780cdd80 con 0x7f4d780cd900 2023-06-23T18:37:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d7e968700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7815cd50 con 0x7f4d78159170 2023-06-23T18:37:27.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d7e968700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7815cfa0 con 0x7f4d780ce620 2023-06-23T18:37:27.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d777fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4d780cd900 0x7f4d780ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d777fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4d780cd900 0x7f4d780ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42452/0 (socket says 172.21.15.79:42452) 2023-06-23T18:37:27.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d777fe700 1 -- 172.21.15.79:0/3425875453 learned_addr learned my addr 172.21.15.79:0/3425875453 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:27.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d77fff700 1 --2- 172.21.15.79:0/3425875453 >> v2:172.21.15.155:3300/0 conn(0x7f4d78159170 0x7f4d780cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d7cf05700 1 --2- 172.21.15.79:0/3425875453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d780ce620 0x7f4d78071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d777fe700 1 -- 172.21.15.79:0/3425875453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d780ce620 msgr2=0x7f4d78071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:27.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d777fe700 1 --2- 172.21.15.79:0/3425875453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d780ce620 0x7f4d78071490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d777fe700 1 -- 172.21.15.79:0/3425875453 >> v2:172.21.15.155:3300/0 conn(0x7f4d78159170 msgr2=0x7f4d780cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:27.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d777fe700 1 --2- 172.21.15.79:0/3425875453 >> v2:172.21.15.155:3300/0 conn(0x7f4d78159170 0x7f4d780cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.987 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d777fe700 1 -- 172.21.15.79:0/3425875453 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7815d280 con 0x7f4d780cd900 2023-06-23T18:37:27.987 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.977+0000 7f4d7cf05700 1 --2- 172.21.15.79:0/3425875453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d780ce620 0x7f4d78071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:27.987 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.978+0000 7f4d777fe700 1 --2- 172.21.15.79:0/3425875453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d780cd900 0x7f4d780ce0e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4d78070aa0 tx=0x7f4d68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f25d2209ba1799 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.987 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.978+0000 7f4d76ffd700 1 -- 172.21.15.79:0/3425875453 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6800b870 con 0x7f4d780cd900 2023-06-23T18:37:27.987 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.978+0000 7f4d76ffd700 1 -- 172.21.15.79:0/3425875453 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6800b9d0 con 0x7f4d780cd900 2023-06-23T18:37:27.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.978+0000 7f4d76ffd700 1 -- 172.21.15.79:0/3425875453 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6800bd60 con 0x7f4d780cd900 2023-06-23T18:37:27.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.978+0000 7f4d7e968700 1 -- 172.21.15.79:0/3425875453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d780cd900 msgr2=0x7f4d780ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.978+0000 7f4d7e968700 1 --2- 172.21.15.79:0/3425875453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d780cd900 0x7f4d780ce0e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4d78070aa0 tx=0x7f4d68009bd0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.978+0000 7f4d7e968700 1 -- 172.21.15.79:0/3425875453 shutdown_connections 2023-06-23T18:37:27.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.978+0000 7f4d7e968700 1 --2- 172.21.15.79:0/3425875453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d780ce620 0x7f4d78071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.978+0000 7f4d7e968700 1 --2- 172.21.15.79:0/3425875453 >> v2:172.21.15.155:3300/0 conn(0x7f4d78159170 0x7f4d780cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.978+0000 7f4d7e968700 1 --2- 172.21.15.79:0/3425875453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d780cd900 0x7f4d780ce0e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.978+0000 7f4d7e968700 1 -- 172.21.15.79:0/3425875453 >> 172.21.15.79:0/3425875453 conn(0x7f4d7806d660 msgr2=0x7f4d7806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:27.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.978+0000 7f4d7e968700 1 -- 172.21.15.79:0/3425875453 shutdown_connections 2023-06-23T18:37:27.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.978+0000 7f4d7e968700 1 -- 172.21.15.79:0/3425875453 wait complete. 2023-06-23T18:37:27.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.979+0000 7f4d7e968700 1 Processor -- start 2023-06-23T18:37:27.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.980+0000 7f4d7e968700 1 -- start start 2023-06-23T18:37:27.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.980+0000 7f4d7e968700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d780cd900 0x7f4d78146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.980+0000 7f4d7e968700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d780ce620 0x7f4d780add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.980+0000 7f4d7e968700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d78159170 0x7f4d780c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.980+0000 7f4d7e968700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d780c8f20 con 0x7f4d780cd900 2023-06-23T18:37:27.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.980+0000 7f4d7e968700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d780c9200 con 0x7f4d780ce620 2023-06-23T18:37:27.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.980+0000 7f4d7e968700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d780c94e0 con 0x7f4d78159170 2023-06-23T18:37:27.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.980+0000 7f4d77fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d780cd900 0x7f4d78146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.980+0000 7f4d77fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d780cd900 0x7f4d78146b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42456/0 (socket says 172.21.15.79:42456) 2023-06-23T18:37:27.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.980+0000 7f4d77fff700 1 -- 172.21.15.79:0/179536335 learned_addr learned my addr 172.21.15.79:0/179536335 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:27.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.981+0000 7f4d777fe700 1 --2- 172.21.15.79:0/179536335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d780ce620 0x7f4d780add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.981+0000 7f4d7cf05700 1 --2- 172.21.15.79:0/179536335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d78159170 0x7f4d780c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.981+0000 7f4d77fff700 1 -- 172.21.15.79:0/179536335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d78159170 msgr2=0x7f4d780c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.981+0000 7f4d77fff700 1 --2- 172.21.15.79:0/179536335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d78159170 0x7f4d780c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.981+0000 7f4d77fff700 1 -- 172.21.15.79:0/179536335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d780ce620 msgr2=0x7f4d780add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:27.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.981+0000 7f4d77fff700 1 --2- 172.21.15.79:0/179536335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d780ce620 0x7f4d780add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:27.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.981+0000 7f4d77fff700 1 -- 172.21.15.79:0/179536335 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d68009550 con 0x7f4d780cd900 2023-06-23T18:37:27.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.981+0000 7f4d77fff700 1 --2- 172.21.15.79:0/179536335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d780cd900 0x7f4d78146b70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00d800 tx=0x7f4d6c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.981+0000 7f4d74ff9700 1 -- 172.21.15.79:0/179536335 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0125f0 con 0x7f4d780cd900 2023-06-23T18:37:27.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.981+0000 7f4d74ff9700 1 -- 172.21.15.79:0/179536335 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c012750 con 0x7f4d780cd900 2023-06-23T18:37:27.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.981+0000 7f4d7e968700 1 -- 172.21.15.79:0/179536335 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d780aba50 con 0x7f4d780cd900 2023-06-23T18:37:27.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.981+0000 7f4d74ff9700 1 -- 172.21.15.79:0/179536335 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c012b00 con 0x7f4d780cd900 2023-06-23T18:37:27.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.981+0000 7f4d7e968700 1 -- 172.21.15.79:0/179536335 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d780ae2a0 con 0x7f4d780cd900 2023-06-23T18:37:27.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.982+0000 7f4d7e968700 1 -- 172.21.15.79:0/179536335 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d780ae4f0 con 0x7f4d780cd900 2023-06-23T18:37:27.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.983+0000 7f4d74ff9700 1 -- 172.21.15.79:0/179536335 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d6c012c60 con 0x7f4d780cd900 2023-06-23T18:37:27.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.983+0000 7f4d74ff9700 1 --2- 172.21.15.79:0/179536335 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4d6406ace0 0x7f4d6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:27.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.983+0000 7f4d74ff9700 1 -- 172.21.15.79:0/179536335 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4d6c0893c0 con 0x7f4d780cd900 2023-06-23T18:37:27.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.983+0000 7f4d777fe700 1 --2- 172.21.15.79:0/179536335 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4d6406ace0 0x7f4d6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:27.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.983+0000 7f4d777fe700 1 --2- 172.21.15.79:0/179536335 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4d6406ace0 0x7f4d6406d190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4d68000c00 tx=0x7f4d68008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:27.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:27.987+0000 7f4d74ff9700 1 -- 172.21.15.79:0/179536335 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4d6c054800 con 0x7f4d780cd900 2023-06-23T18:37:28.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.176+0000 7f4d7e968700 1 -- 172.21.15.79:0/179536335 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d7815d840 con 0x7f4d6406ace0 2023-06-23T18:37:28.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.176+0000 7f4d74ff9700 1 -- 172.21.15.79:0/179536335 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4d6800bb40 con 0x7f4d6406ace0 2023-06-23T18:37:28.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.178+0000 7f4d7e968700 1 -- 172.21.15.79:0/179536335 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4d6406ace0 msgr2=0x7f4d6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.178+0000 7f4d7e968700 1 --2- 172.21.15.79:0/179536335 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4d6406ace0 0x7f4d6406d190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4d68000c00 tx=0x7f4d68008030 comp rx=0 tx=0).stop 2023-06-23T18:37:28.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.178+0000 7f4d7e968700 1 -- 172.21.15.79:0/179536335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d780cd900 msgr2=0x7f4d78146b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.178+0000 7f4d7e968700 1 --2- 172.21.15.79:0/179536335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d780cd900 0x7f4d78146b70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00d800 tx=0x7f4d6c00db10 comp rx=0 tx=0).stop 2023-06-23T18:37:28.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.178+0000 7f4d7e968700 1 -- 172.21.15.79:0/179536335 shutdown_connections 2023-06-23T18:37:28.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.178+0000 7f4d7e968700 1 --2- 172.21.15.79:0/179536335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d78159170 0x7f4d780c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.178+0000 7f4d7e968700 1 --2- 172.21.15.79:0/179536335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d780cd900 0x7f4d78146b70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.178+0000 7f4d7e968700 1 --2- 172.21.15.79:0/179536335 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f4d6406ace0 0x7f4d6406d190 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.178+0000 7f4d7e968700 1 --2- 172.21.15.79:0/179536335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d780ce620 0x7f4d780add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.178+0000 7f4d7e968700 1 -- 172.21.15.79:0/179536335 >> 172.21.15.79:0/179536335 conn(0x7f4d7806d660 msgr2=0x7f4d780b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:28.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.178+0000 7f4d7e968700 1 -- 172.21.15.79:0/179536335 shutdown_connections 2023-06-23T18:37:28.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.178+0000 7f4d7e968700 1 -- 172.21.15.79:0/179536335 wait complete. 2023-06-23T18:37:28.186 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 5 to deep-scrub 2023-06-23T18:37:28.195 INFO:tasks.ceph:Scrubbing osd.6 2023-06-23T18:37:28.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-06-23T18:37:28.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.313+0000 7f62a3888700 1 Processor -- start 2023-06-23T18:37:28.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.314+0000 7f62a3888700 1 -- start start 2023-06-23T18:37:28.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.314+0000 7f62a3888700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f629c0cb870 0x7f629c160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.314+0000 7f62a3888700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f629c069e90 0x7f629c06a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.314+0000 7f62a3888700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629c160f50 con 0x7f629c0cf1f0 2023-06-23T18:37:28.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.314+0000 7f62a3888700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629c06a870 con 0x7f629c069e90 2023-06-23T18:37:28.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.314+0000 7f62a3888700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629c15cac0 con 0x7f629c0cb870 2023-06-23T18:37:28.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.314+0000 7f62a1e25700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f629c069e90 0x7f629c06a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.315+0000 7f62a1e25700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f629c069e90 0x7f629c06a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42468/0 (socket says 172.21.15.79:42468) 2023-06-23T18:37:28.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.315+0000 7f62a1e25700 1 -- 172.21.15.79:0/1195908439 learned_addr learned my addr 172.21.15.79:0/1195908439 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:28.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.315+0000 7f62a1624700 1 --2- 172.21.15.79:0/1195908439 >> v2:172.21.15.155:3300/0 conn(0x7f629c0cb870 0x7f629c160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.315+0000 7f62a1e25700 1 -- 172.21.15.79:0/1195908439 >> v2:172.21.15.155:3300/0 conn(0x7f629c0cb870 msgr2=0x7f629c160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:28.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.315+0000 7f62a1e25700 1 --2- 172.21.15.79:0/1195908439 >> v2:172.21.15.155:3300/0 conn(0x7f629c0cb870 0x7f629c160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.315+0000 7f62a1e25700 1 -- 172.21.15.79:0/1195908439 >> v1:172.21.15.155:6789/0 conn(0x7f629c0cf1f0 legacy=0x7f629c15d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:28.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.315+0000 7f62a1e25700 1 -- 172.21.15.79:0/1195908439 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c0cd410 con 0x7f629c069e90 2023-06-23T18:37:28.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.315+0000 7f62a1624700 1 --2- 172.21.15.79:0/1195908439 >> v2:172.21.15.155:3300/0 conn(0x7f629c0cb870 0x7f629c160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:28.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.315+0000 7f62a1e25700 1 --2- 172.21.15.79:0/1195908439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f629c069e90 0x7f629c06a330 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f629c0d0940 tx=0x7f629400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25617b21b05b392 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:28.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.315+0000 7f6293fff700 1 -- 172.21.15.79:0/1195908439 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629400e070 con 0x7f629c069e90 2023-06-23T18:37:28.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.315+0000 7f6293fff700 1 -- 172.21.15.79:0/1195908439 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6294003a20 con 0x7f629c069e90 2023-06-23T18:37:28.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.315+0000 7f6293fff700 1 -- 172.21.15.79:0/1195908439 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6294003dd0 con 0x7f629c069e90 2023-06-23T18:37:28.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.316+0000 7f62a3888700 1 -- 172.21.15.79:0/1195908439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f629c069e90 msgr2=0x7f629c06a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.316+0000 7f62a3888700 1 --2- 172.21.15.79:0/1195908439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f629c069e90 0x7f629c06a330 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f629c0d0940 tx=0x7f629400b330 comp rx=0 tx=0).stop 2023-06-23T18:37:28.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.316+0000 7f62a3888700 1 -- 172.21.15.79:0/1195908439 shutdown_connections 2023-06-23T18:37:28.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.316+0000 7f62a3888700 1 --2- 172.21.15.79:0/1195908439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f629c069e90 0x7f629c06a330 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.316+0000 7f62a3888700 1 --2- 172.21.15.79:0/1195908439 >> v2:172.21.15.155:3300/0 conn(0x7f629c0cb870 0x7f629c160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.316+0000 7f62a3888700 1 -- 172.21.15.79:0/1195908439 >> 172.21.15.79:0/1195908439 conn(0x7f629c06e200 msgr2=0x7f629c0cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:28.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.316+0000 7f62a3888700 1 -- 172.21.15.79:0/1195908439 shutdown_connections 2023-06-23T18:37:28.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.316+0000 7f62a3888700 1 -- 172.21.15.79:0/1195908439 wait complete. 2023-06-23T18:37:28.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.317+0000 7f62a3888700 1 Processor -- start 2023-06-23T18:37:28.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.317+0000 7f62a3888700 1 -- start start 2023-06-23T18:37:28.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.318+0000 7f62a3888700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f629c069e90 0x7f629c0ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.318+0000 7f62a3888700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f629c0cb870 0x7f629c157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.318+0000 7f62a3888700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f629c0cf1f0 0x7f629c0b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.318+0000 7f62a3888700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c1e6a80 con 0x7f629c0cf1f0 2023-06-23T18:37:28.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.318+0000 7f62a3888700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c1e5e00 con 0x7f629c069e90 2023-06-23T18:37:28.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.318+0000 7f62a3888700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c1e60e0 con 0x7f629c0cb870 2023-06-23T18:37:28.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.318+0000 7f62a1e25700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f629c0cf1f0 0x7f629c0b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.318+0000 7f62a1e25700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f629c0cf1f0 0x7f629c0b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42474/0 (socket says 172.21.15.79:42474) 2023-06-23T18:37:28.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.318+0000 7f62a1e25700 1 -- 172.21.15.79:0/117458127 learned_addr learned my addr 172.21.15.79:0/117458127 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:28.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.318+0000 7f62a1624700 1 --2- 172.21.15.79:0/117458127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f629c069e90 0x7f629c0ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.318+0000 7f62a0e23700 1 --2- 172.21.15.79:0/117458127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f629c0cb870 0x7f629c157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.319+0000 7f62a1e25700 1 -- 172.21.15.79:0/117458127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f629c0cb870 msgr2=0x7f629c157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.319+0000 7f62a1e25700 1 --2- 172.21.15.79:0/117458127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f629c0cb870 0x7f629c157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.319+0000 7f62a1e25700 1 -- 172.21.15.79:0/117458127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f629c069e90 msgr2=0x7f629c0ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.319+0000 7f62a1e25700 1 --2- 172.21.15.79:0/117458127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f629c069e90 0x7f629c0ad660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.319+0000 7f62a1e25700 1 -- 172.21.15.79:0/117458127 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6294009b30 con 0x7f629c0cf1f0 2023-06-23T18:37:28.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.319+0000 7f62a1e25700 1 --2- 172.21.15.79:0/117458127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f629c0cf1f0 0x7f629c0b4490 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6294015040 tx=0x7f6294004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:28.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.319+0000 7f6291ffb700 1 -- 172.21.15.79:0/117458127 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62940092e0 con 0x7f629c0cf1f0 2023-06-23T18:37:28.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.319+0000 7f6291ffb700 1 -- 172.21.15.79:0/117458127 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6294004240 con 0x7f629c0cf1f0 2023-06-23T18:37:28.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.319+0000 7f62a3888700 1 -- 172.21.15.79:0/117458127 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629c1e63c0 con 0x7f629c0cf1f0 2023-06-23T18:37:28.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.319+0000 7f6291ffb700 1 -- 172.21.15.79:0/117458127 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62940045f0 con 0x7f629c0cf1f0 2023-06-23T18:37:28.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.319+0000 7f62a3888700 1 -- 172.21.15.79:0/117458127 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629c0adba0 con 0x7f629c0cf1f0 2023-06-23T18:37:28.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.320+0000 7f62a3888700 1 -- 172.21.15.79:0/117458127 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f629c0cdc20 con 0x7f629c0cf1f0 2023-06-23T18:37:28.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.321+0000 7f6291ffb700 1 -- 172.21.15.79:0/117458127 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6294019070 con 0x7f629c0cf1f0 2023-06-23T18:37:28.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.321+0000 7f6291ffb700 1 --2- 172.21.15.79:0/117458127 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f628006ace0 0x7f628006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.321+0000 7f6291ffb700 1 -- 172.21.15.79:0/117458127 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6294089e40 con 0x7f629c0cf1f0 2023-06-23T18:37:28.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.321+0000 7f62a1624700 1 --2- 172.21.15.79:0/117458127 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f628006ace0 0x7f628006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.321+0000 7f6291ffb700 1 --2- 172.21.15.79:0/117458127 >> [v2:172.21.15.155:6801/1282622313,v1:172.21.15.155:6802/1282622313] conn(0x7f6280070fe0 0x7f62800733f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.321+0000 7f6291ffb700 1 -- 172.21.15.79:0/117458127 --> [v2:172.21.15.155:6801/1282622313,v1:172.21.15.155:6802/1282622313] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6280073ac0 con 0x7f6280070fe0 2023-06-23T18:37:28.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.321+0000 7f6291ffb700 1 -- 172.21.15.79:0/117458127 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f629408c4e0 con 0x7f629c0cf1f0 2023-06-23T18:37:28.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.321+0000 7f62a1624700 1 --2- 172.21.15.79:0/117458127 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f628006ace0 0x7f628006d190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f628c006fd0 tx=0x7f628c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:28.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.322+0000 7f62a0e23700 1 --2- 172.21.15.79:0/117458127 >> [v2:172.21.15.155:6801/1282622313,v1:172.21.15.155:6802/1282622313] conn(0x7f6280070fe0 0x7f62800733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.322+0000 7f62a0e23700 1 --2- 172.21.15.79:0/117458127 >> [v2:172.21.15.155:6801/1282622313,v1:172.21.15.155:6802/1282622313] conn(0x7f6280070fe0 0x7f62800733f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:28.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.323+0000 7f6291ffb700 1 -- 172.21.15.79:0/117458127 <== osd.6 v2:172.21.15.155:6801/1282622313 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6280073ac0 con 0x7f6280070fe0 2023-06-23T18:37:28.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.337+0000 7f62a3888700 1 -- 172.21.15.79:0/117458127 --> [v2:172.21.15.155:6801/1282622313,v1:172.21.15.155:6802/1282622313] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f629c15d040 con 0x7f6280070fe0 2023-06-23T18:37:28.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.337+0000 7f6291ffb700 1 -- 172.21.15.79:0/117458127 <== osd.6 v2:172.21.15.155:6801/1282622313 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6298003b50 con 0x7f6280070fe0 2023-06-23T18:37:28.344 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-23T18:37:28.344 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-23T18:37:28.344 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-23T18:37:28.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.338+0000 7f62a3888700 1 -- 172.21.15.79:0/117458127 >> [v2:172.21.15.155:6801/1282622313,v1:172.21.15.155:6802/1282622313] conn(0x7f6280070fe0 msgr2=0x7f62800733f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.338+0000 7f62a3888700 1 --2- 172.21.15.79:0/117458127 >> [v2:172.21.15.155:6801/1282622313,v1:172.21.15.155:6802/1282622313] conn(0x7f6280070fe0 0x7f62800733f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.338+0000 7f62a3888700 1 -- 172.21.15.79:0/117458127 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f628006ace0 msgr2=0x7f628006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.338+0000 7f62a3888700 1 --2- 172.21.15.79:0/117458127 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f628006ace0 0x7f628006d190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f628c006fd0 tx=0x7f628c008040 comp rx=0 tx=0).stop 2023-06-23T18:37:28.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.338+0000 7f62a3888700 1 -- 172.21.15.79:0/117458127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f629c0cf1f0 msgr2=0x7f629c0b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.338+0000 7f62a3888700 1 --2- 172.21.15.79:0/117458127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f629c0cf1f0 0x7f629c0b4490 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6294015040 tx=0x7f6294004ad0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.338+0000 7f62a1e25700 1 -- 172.21.15.79:0/117458127 reap_dead start 2023-06-23T18:37:28.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.338+0000 7f62a3888700 1 -- 172.21.15.79:0/117458127 shutdown_connections 2023-06-23T18:37:28.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.338+0000 7f62a3888700 1 -- 172.21.15.79:0/117458127 >> 172.21.15.79:0/117458127 conn(0x7f629c06e200 msgr2=0x7f629c0b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:28.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.338+0000 7f62a3888700 1 -- 172.21.15.79:0/117458127 shutdown_connections 2023-06-23T18:37:28.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.338+0000 7f62a3888700 1 -- 172.21.15.79:0/117458127 wait complete. 2023-06-23T18:37:28.354 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-06-23T18:37:28.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.470+0000 7f8a7dad7700 1 Processor -- start 2023-06-23T18:37:28.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.470+0000 7f8a7dad7700 1 -- start start 2023-06-23T18:37:28.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.471+0000 7f8a7dad7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a78159170 0x7f8a780cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.471+0000 7f8a7dad7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7815cd50 con 0x7f8a780ce7f0 2023-06-23T18:37:28.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.471+0000 7f8a7dad7700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7815cfa0 con 0x7f8a780cd900 2023-06-23T18:37:28.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.471+0000 7f8a7dad7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a780cdd70 con 0x7f8a78159170 2023-06-23T18:37:28.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.471+0000 7f8a77fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f8a780ce7f0 0x7f8a780cc1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45526/0 (socket says 172.21.15.79:45526) 2023-06-23T18:37:28.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.471+0000 7f8a77fff700 1 -- 172.21.15.79:0/2400841820 learned_addr learned my addr 172.21.15.79:0/2400841820 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:28.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.472+0000 7f8a777fe700 1 --2- 172.21.15.79:0/2400841820 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a78159170 0x7f8a780cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.472+0000 7f8a767fc700 1 -- 172.21.15.79:0/2400841820 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2518485141 0 0) 0x7f8a7815cd50 con 0x7f8a780ce7f0 2023-06-23T18:37:28.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.472+0000 7f8a767fc700 1 -- 172.21.15.79:0/2400841820 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a600035b0 con 0x7f8a780ce7f0 2023-06-23T18:37:28.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.472+0000 7f8a767fc700 1 -- 172.21.15.79:0/2400841820 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790600504 0 0) 0x7f8a7815cfa0 con 0x7f8a780cd900 2023-06-23T18:37:28.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.472+0000 7f8a767fc700 1 -- 172.21.15.79:0/2400841820 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a7815cd50 con 0x7f8a780cd900 2023-06-23T18:37:28.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.472+0000 7f8a777fe700 1 -- 172.21.15.79:0/2400841820 >> v1:172.21.15.155:6789/0 conn(0x7f8a780cd900 legacy=0x7f8a780ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.472+0000 7f8a777fe700 1 -- 172.21.15.79:0/2400841820 >> v1:172.21.15.79:6789/0 conn(0x7f8a780ce7f0 legacy=0x7f8a780cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.472+0000 7f8a767fc700 1 -- 172.21.15.79:0/2400841820 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 808261588 0 0) 0x7f8a600035b0 con 0x7f8a780ce7f0 2023-06-23T18:37:28.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.472+0000 7f8a777fe700 1 -- 172.21.15.79:0/2400841820 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7815d280 con 0x7f8a78159170 2023-06-23T18:37:28.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.472+0000 7f8a777fe700 1 --2- 172.21.15.79:0/2400841820 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a78159170 0x7f8a780cd3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009870 tx=0x7f8a6c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b89b3a05cb92a249 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:28.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.472+0000 7f8a767fc700 1 -- 172.21.15.79:0/2400841820 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a700019f0 con 0x7f8a78159170 2023-06-23T18:37:28.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.472+0000 7f8a767fc700 1 -- 172.21.15.79:0/2400841820 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c004030 con 0x7f8a78159170 2023-06-23T18:37:28.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.472+0000 7f8a767fc700 1 -- 172.21.15.79:0/2400841820 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c0043c0 con 0x7f8a78159170 2023-06-23T18:37:28.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.473+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2400841820 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a78159170 msgr2=0x7f8a780cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.473+0000 7f8a7dad7700 1 --2- 172.21.15.79:0/2400841820 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a78159170 0x7f8a780cd3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009870 tx=0x7f8a6c009b80 comp rx=0 tx=0).stop 2023-06-23T18:37:28.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.473+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2400841820 shutdown_connections 2023-06-23T18:37:28.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.473+0000 7f8a7dad7700 1 --2- 172.21.15.79:0/2400841820 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a78159170 0x7f8a780cd3c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.473+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2400841820 >> 172.21.15.79:0/2400841820 conn(0x7f8a7806d660 msgr2=0x7f8a7806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:28.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.473+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2400841820 shutdown_connections 2023-06-23T18:37:28.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.473+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2400841820 wait complete. 2023-06-23T18:37:28.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.474+0000 7f8a7dad7700 1 Processor -- start 2023-06-23T18:37:28.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.474+0000 7f8a7dad7700 1 -- start start 2023-06-23T18:37:28.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.475+0000 7f8a7dad7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a780cd900 0x7f8a780abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.475+0000 7f8a7dad7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a780ce7f0 0x7f8a78157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.475+0000 7f8a7dad7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a78159170 0x7f8a781e4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.475+0000 7f8a7dad7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a780c61d0 con 0x7f8a78159170 2023-06-23T18:37:28.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.475+0000 7f8a7dad7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a781f21a0 con 0x7f8a780cd900 2023-06-23T18:37:28.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.475+0000 7f8a7dad7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a780c5460 con 0x7f8a780ce7f0 2023-06-23T18:37:28.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.475+0000 7f8a77fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a78159170 0x7f8a781e4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.475+0000 7f8a77fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a78159170 0x7f8a781e4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42490/0 (socket says 172.21.15.79:42490) 2023-06-23T18:37:28.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.475+0000 7f8a77fff700 1 -- 172.21.15.79:0/2972883477 learned_addr learned my addr 172.21.15.79:0/2972883477 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:28.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.475+0000 7f8a77fff700 1 -- 172.21.15.79:0/2972883477 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a780ce7f0 msgr2=0x7f8a78157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.475+0000 7f8a77fff700 1 --2- 172.21.15.79:0/2972883477 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a780ce7f0 0x7f8a78157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.475+0000 7f8a77fff700 1 -- 172.21.15.79:0/2972883477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a780cd900 msgr2=0x7f8a780abdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.475+0000 7f8a77fff700 1 --2- 172.21.15.79:0/2972883477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a780cd900 0x7f8a780abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.475+0000 7f8a77fff700 1 -- 172.21.15.79:0/2972883477 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c0094a0 con 0x7f8a78159170 2023-06-23T18:37:28.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.476+0000 7f8a77fff700 1 --2- 172.21.15.79:0/2972883477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a78159170 0x7f8a781e4bb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8a70001870 tx=0x7f8a7000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:28.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.476+0000 7f8a5ffff700 1 -- 172.21.15.79:0/2972883477 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7000ad80 con 0x7f8a78159170 2023-06-23T18:37:28.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.476+0000 7f8a5ffff700 1 -- 172.21.15.79:0/2972883477 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a700070d0 con 0x7f8a78159170 2023-06-23T18:37:28.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.476+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2972883477 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a780c5930 con 0x7f8a78159170 2023-06-23T18:37:28.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.476+0000 7f8a5ffff700 1 -- 172.21.15.79:0/2972883477 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70007480 con 0x7f8a78159170 2023-06-23T18:37:28.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.476+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2972883477 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a780c5ef0 con 0x7f8a78159170 2023-06-23T18:37:28.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.477+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2972883477 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a781f2310 con 0x7f8a78159170 2023-06-23T18:37:28.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.477+0000 7f8a5ffff700 1 -- 172.21.15.79:0/2972883477 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8a700075e0 con 0x7f8a78159170 2023-06-23T18:37:28.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.477+0000 7f8a5ffff700 1 --2- 172.21.15.79:0/2972883477 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8a6406ace0 0x7f8a6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.478+0000 7f8a5ffff700 1 -- 172.21.15.79:0/2972883477 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8a70089180 con 0x7f8a78159170 2023-06-23T18:37:28.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.478+0000 7f8a777fe700 1 --2- 172.21.15.79:0/2972883477 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8a6406ace0 0x7f8a6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.478+0000 7f8a777fe700 1 --2- 172.21.15.79:0/2972883477 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8a6406ace0 0x7f8a6406d190 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c000c00 tx=0x7f8a6c010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:28.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.482+0000 7f8a5ffff700 1 -- 172.21.15.79:0/2972883477 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8a70054530 con 0x7f8a78159170 2023-06-23T18:37:28.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.669+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2972883477 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a7815d840 con 0x7f8a6406ace0 2023-06-23T18:37:28.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.669+0000 7f8a5ffff700 1 -- 172.21.15.79:0/2972883477 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8a6c01f070 con 0x7f8a6406ace0 2023-06-23T18:37:28.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.671+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2972883477 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8a6406ace0 msgr2=0x7f8a6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.671+0000 7f8a7dad7700 1 --2- 172.21.15.79:0/2972883477 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8a6406ace0 0x7f8a6406d190 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c000c00 tx=0x7f8a6c010000 comp rx=0 tx=0).stop 2023-06-23T18:37:28.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.671+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2972883477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a78159170 msgr2=0x7f8a781e4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.671+0000 7f8a7dad7700 1 --2- 172.21.15.79:0/2972883477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a78159170 0x7f8a781e4bb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8a70001870 tx=0x7f8a7000a340 comp rx=0 tx=0).stop 2023-06-23T18:37:28.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.671+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2972883477 shutdown_connections 2023-06-23T18:37:28.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.671+0000 7f8a7dad7700 1 --2- 172.21.15.79:0/2972883477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a78159170 0x7f8a781e4bb0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.671+0000 7f8a7dad7700 1 --2- 172.21.15.79:0/2972883477 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f8a6406ace0 0x7f8a6406d190 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.671+0000 7f8a7dad7700 1 --2- 172.21.15.79:0/2972883477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a780cd900 0x7f8a780abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.671+0000 7f8a7dad7700 1 --2- 172.21.15.79:0/2972883477 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a780ce7f0 0x7f8a78157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.671+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2972883477 >> 172.21.15.79:0/2972883477 conn(0x7f8a7806d660 msgr2=0x7f8a780cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:28.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.671+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2972883477 shutdown_connections 2023-06-23T18:37:28.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.671+0000 7f8a7dad7700 1 -- 172.21.15.79:0/2972883477 wait complete. 2023-06-23T18:37:28.679 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 6 to deep-scrub 2023-06-23T18:37:28.688 INFO:tasks.ceph:Scrubbing osd.7 2023-06-23T18:37:28.688 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-06-23T18:37:28.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.807+0000 7f917dfa2700 1 Processor -- start 2023-06-23T18:37:28.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.807+0000 7f917dfa2700 1 -- start start 2023-06-23T18:37:28.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.808+0000 7f917dfa2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f91780cb870 0x7f9178160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.808+0000 7f917dfa2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91780cf1f0 0x7f917815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.808+0000 7f917dfa2700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9178160f50 con 0x7f9178069e90 2023-06-23T18:37:28.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.808+0000 7f917dfa2700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f917815cac0 con 0x7f91780cb870 2023-06-23T18:37:28.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.808+0000 7f917dfa2700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91780cd410 con 0x7f91780cf1f0 2023-06-23T18:37:28.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.808+0000 7f91777fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f91780cb870 0x7f9178160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.808+0000 7f9177fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9178069e90 0x7f917806eeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45536/0 (socket says 172.21.15.79:45536) 2023-06-23T18:37:28.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.808+0000 7f91777fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f91780cb870 0x7f9178160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42494/0 (socket says 172.21.15.79:42494) 2023-06-23T18:37:28.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.808+0000 7f9177fff700 1 -- 172.21.15.79:0/2887904469 learned_addr learned my addr 172.21.15.79:0/2887904469 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:28.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f9176ffd700 1 --2- 172.21.15.79:0/2887904469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91780cf1f0 0x7f917815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f91777fe700 1 -- 172.21.15.79:0/2887904469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91780cf1f0 msgr2=0x7f917815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:28.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f91777fe700 1 --2- 172.21.15.79:0/2887904469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91780cf1f0 0x7f917815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f91777fe700 1 -- 172.21.15.79:0/2887904469 >> v1:172.21.15.79:6789/0 conn(0x7f9178069e90 legacy=0x7f917806eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f91767fc700 1 -- 172.21.15.79:0/2887904469 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745163367 0 0) 0x7f9170001490 con 0x7f9178069e90 2023-06-23T18:37:28.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f91777fe700 1 -- 172.21.15.79:0/2887904469 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917815d040 con 0x7f91780cb870 2023-06-23T18:37:28.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f9176ffd700 1 --2- 172.21.15.79:0/2887904469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91780cf1f0 0x7f917815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:28.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f91777fe700 1 --2- 172.21.15.79:0/2887904469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91780cb870 0x7f9178160a10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f916c009870 tx=0x7f916c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403464995ea0a7ae server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:28.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f91767fc700 1 -- 172.21.15.79:0/2887904469 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c011830 con 0x7f91780cb870 2023-06-23T18:37:28.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f91767fc700 1 -- 172.21.15.79:0/2887904469 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c011990 con 0x7f91780cb870 2023-06-23T18:37:28.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f91767fc700 1 -- 172.21.15.79:0/2887904469 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c011d20 con 0x7f91780cb870 2023-06-23T18:37:28.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f917dfa2700 1 -- 172.21.15.79:0/2887904469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91780cb870 msgr2=0x7f9178160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f917dfa2700 1 --2- 172.21.15.79:0/2887904469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91780cb870 0x7f9178160a10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f916c009870 tx=0x7f916c009b80 comp rx=0 tx=0).stop 2023-06-23T18:37:28.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f917dfa2700 1 -- 172.21.15.79:0/2887904469 shutdown_connections 2023-06-23T18:37:28.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f917dfa2700 1 --2- 172.21.15.79:0/2887904469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91780cb870 0x7f9178160a10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f917dfa2700 1 --2- 172.21.15.79:0/2887904469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91780cf1f0 0x7f917815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.809+0000 7f917dfa2700 1 -- 172.21.15.79:0/2887904469 >> 172.21.15.79:0/2887904469 conn(0x7f917806e200 msgr2=0x7f91780cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:28.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.810+0000 7f917dfa2700 1 -- 172.21.15.79:0/2887904469 shutdown_connections 2023-06-23T18:37:28.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.810+0000 7f917dfa2700 1 -- 172.21.15.79:0/2887904469 wait complete. 2023-06-23T18:37:28.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.811+0000 7f917dfa2700 1 Processor -- start 2023-06-23T18:37:28.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.811+0000 7f917dfa2700 1 -- start start 2023-06-23T18:37:28.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f917dfa2700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9178069e90 0x7f91781e9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f917dfa2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91780cb870 0x7f91781f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f917dfa2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91780cf1f0 0x7f91781f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f917dfa2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91780b2700 con 0x7f91780cb870 2023-06-23T18:37:28.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f917dfa2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91781e7d20 con 0x7f91780cf1f0 2023-06-23T18:37:28.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f917dfa2700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91781e8000 con 0x7f9178069e90 2023-06-23T18:37:28.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f9176ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91780cb870 0x7f91781f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f9176ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91780cb870 0x7f91781f5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42500/0 (socket says 172.21.15.79:42500) 2023-06-23T18:37:28.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f9176ffd700 1 -- 172.21.15.79:0/472501991 learned_addr learned my addr 172.21.15.79:0/472501991 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:28.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f91777fe700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9178069e90 0x7f91781e9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f9177fff700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91780cf1f0 0x7f91781f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f9176ffd700 1 -- 172.21.15.79:0/472501991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9178069e90 msgr2=0x7f91781e9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f9176ffd700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9178069e90 0x7f91781e9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f9176ffd700 1 -- 172.21.15.79:0/472501991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91780cf1f0 msgr2=0x7f91781f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f9176ffd700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91780cf1f0 0x7f91781f5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f9176ffd700 1 -- 172.21.15.79:0/472501991 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0094a0 con 0x7f91780cb870 2023-06-23T18:37:28.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f91777fe700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9178069e90 0x7f91781e9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:28.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f9177fff700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91780cf1f0 0x7f91781f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:28.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f9176ffd700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91780cb870 0x7f91781f5990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f916800c750 tx=0x7f916800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:28.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f915ffff700 1 -- 172.21.15.79:0/472501991 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168011800 con 0x7f91780cb870 2023-06-23T18:37:28.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f915ffff700 1 -- 172.21.15.79:0/472501991 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9168011960 con 0x7f91780cb870 2023-06-23T18:37:28.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f917dfa2700 1 -- 172.21.15.79:0/472501991 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91781e82e0 con 0x7f91780cb870 2023-06-23T18:37:28.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f915ffff700 1 -- 172.21.15.79:0/472501991 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168011cd0 con 0x7f91780cb870 2023-06-23T18:37:28.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.812+0000 7f917dfa2700 1 -- 172.21.15.79:0/472501991 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91781e7920 con 0x7f91780cb870 2023-06-23T18:37:28.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.813+0000 7f917dfa2700 1 -- 172.21.15.79:0/472501991 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f917815da10 con 0x7f91780cb870 2023-06-23T18:37:28.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.814+0000 7f915ffff700 1 -- 172.21.15.79:0/472501991 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9168021350 con 0x7f91780cb870 2023-06-23T18:37:28.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.814+0000 7f915ffff700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f916406ace0 0x7f916406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.814+0000 7f915ffff700 1 -- 172.21.15.79:0/472501991 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f91680890d0 con 0x7f91780cb870 2023-06-23T18:37:28.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.814+0000 7f91777fe700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f916406ace0 0x7f916406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.814+0000 7f915ffff700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.155:6824/3369626517,v1:172.21.15.155:6825/3369626517] conn(0x7f9164071150 0x7f9164073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.814+0000 7f915ffff700 1 -- 172.21.15.79:0/472501991 --> [v2:172.21.15.155:6824/3369626517,v1:172.21.15.155:6825/3369626517] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9164073c30 con 0x7f9164071150 2023-06-23T18:37:28.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.814+0000 7f915ffff700 1 -- 172.21.15.79:0/472501991 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f916808b770 con 0x7f91780cb870 2023-06-23T18:37:28.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.814+0000 7f9177fff700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.155:6824/3369626517,v1:172.21.15.155:6825/3369626517] conn(0x7f9164071150 0x7f9164073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.814+0000 7f91777fe700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f916406ace0 0x7f916406d190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f916c0090b0 tx=0x7f916c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:28.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.815+0000 7f9177fff700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.155:6824/3369626517,v1:172.21.15.155:6825/3369626517] conn(0x7f9164071150 0x7f9164073560 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:28.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.816+0000 7f915ffff700 1 -- 172.21.15.79:0/472501991 <== osd.7 v2:172.21.15.155:6824/3369626517 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9164073c30 con 0x7f9164071150 2023-06-23T18:37:28.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.829+0000 7f917dfa2700 1 -- 172.21.15.79:0/472501991 --> [v2:172.21.15.155:6824/3369626517,v1:172.21.15.155:6825/3369626517] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f91780cdc20 con 0x7f9164071150 2023-06-23T18:37:28.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.830+0000 7f915ffff700 1 -- 172.21.15.79:0/472501991 <== osd.7 v2:172.21.15.155:6824/3369626517 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f91700071b0 con 0x7f9164071150 2023-06-23T18:37:28.836 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-23T18:37:28.837 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-23T18:37:28.837 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-23T18:37:28.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.830+0000 7f917dfa2700 1 -- 172.21.15.79:0/472501991 >> [v2:172.21.15.155:6824/3369626517,v1:172.21.15.155:6825/3369626517] conn(0x7f9164071150 msgr2=0x7f9164073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.830+0000 7f917dfa2700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.155:6824/3369626517,v1:172.21.15.155:6825/3369626517] conn(0x7f9164071150 0x7f9164073560 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.830+0000 7f917dfa2700 1 -- 172.21.15.79:0/472501991 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f916406ace0 msgr2=0x7f916406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.830+0000 7f917dfa2700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f916406ace0 0x7f916406d190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f916c0090b0 tx=0x7f916c008030 comp rx=0 tx=0).stop 2023-06-23T18:37:28.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.830+0000 7f917dfa2700 1 -- 172.21.15.79:0/472501991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91780cb870 msgr2=0x7f91781f5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.830+0000 7f917dfa2700 1 --2- 172.21.15.79:0/472501991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91780cb870 0x7f91781f5990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f916800c750 tx=0x7f916800cb10 comp rx=0 tx=0).stop 2023-06-23T18:37:28.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.830+0000 7f9177fff700 1 -- 172.21.15.79:0/472501991 reap_dead start 2023-06-23T18:37:28.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.830+0000 7f917dfa2700 1 -- 172.21.15.79:0/472501991 shutdown_connections 2023-06-23T18:37:28.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.830+0000 7f917dfa2700 1 -- 172.21.15.79:0/472501991 >> 172.21.15.79:0/472501991 conn(0x7f917806e200 msgr2=0x7f917806eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:28.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.830+0000 7f917dfa2700 1 -- 172.21.15.79:0/472501991 shutdown_connections 2023-06-23T18:37:28.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.830+0000 7f917dfa2700 1 -- 172.21.15.79:0/472501991 wait complete. 2023-06-23T18:37:28.847 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-06-23T18:37:28.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.962+0000 7f8496230700 1 Processor -- start 2023-06-23T18:37:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.962+0000 7f8496230700 1 -- start start 2023-06-23T18:37:28.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.963+0000 7f8496230700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8490159170 0x7f84900cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.963+0000 7f8496230700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84900ce7f0 0x7f84900cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.963+0000 7f8496230700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f849015cd50 con 0x7f84900cd900 2023-06-23T18:37:28.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.963+0000 7f8496230700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84900cdd80 con 0x7f8490159170 2023-06-23T18:37:28.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.963+0000 7f8496230700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849015cfa0 con 0x7f84900ce7f0 2023-06-23T18:37:28.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.963+0000 7f848f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8490159170 0x7f84900cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.963+0000 7f848f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8490159170 0x7f84900cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42508/0 (socket says 172.21.15.79:42508) 2023-06-23T18:37:28.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.963+0000 7f848effd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f84900cd900 0x7f84900ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45550/0 (socket says 172.21.15.79:45550) 2023-06-23T18:37:28.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.963+0000 7f848f7fe700 1 -- 172.21.15.79:0/4188809225 learned_addr learned my addr 172.21.15.79:0/4188809225 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:28.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f848ffff700 1 --2- 172.21.15.79:0/4188809225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84900ce7f0 0x7f84900cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f848f7fe700 1 -- 172.21.15.79:0/4188809225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84900ce7f0 msgr2=0x7f84900cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:28.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f848f7fe700 1 --2- 172.21.15.79:0/4188809225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84900ce7f0 0x7f84900cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f848e7fc700 1 -- 172.21.15.79:0/4188809225 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786982048 0 0) 0x7f849015cd50 con 0x7f84900cd900 2023-06-23T18:37:28.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f848f7fe700 1 -- 172.21.15.79:0/4188809225 >> v1:172.21.15.79:6789/0 conn(0x7f84900cd900 legacy=0x7f84900ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f848f7fe700 1 -- 172.21.15.79:0/4188809225 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849015d280 con 0x7f8490159170 2023-06-23T18:37:28.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f848ffff700 1 --2- 172.21.15.79:0/4188809225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84900ce7f0 0x7f84900cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:28.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f848f7fe700 1 --2- 172.21.15.79:0/4188809225 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8490159170 0x7f84900cd3c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f847800bb90 tx=0x7f847800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9377b6cee558f7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:28.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f848e7fc700 1 -- 172.21.15.79:0/4188809225 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847800dd50 con 0x7f8490159170 2023-06-23T18:37:28.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f848e7fc700 1 -- 172.21.15.79:0/4188809225 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8478010470 con 0x7f8490159170 2023-06-23T18:37:28.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f848e7fc700 1 -- 172.21.15.79:0/4188809225 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478010800 con 0x7f8490159170 2023-06-23T18:37:28.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f8496230700 1 -- 172.21.15.79:0/4188809225 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8490159170 msgr2=0x7f84900cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f8496230700 1 --2- 172.21.15.79:0/4188809225 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8490159170 0x7f84900cd3c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f847800bb90 tx=0x7f847800bea0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f8496230700 1 -- 172.21.15.79:0/4188809225 shutdown_connections 2023-06-23T18:37:28.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f8496230700 1 --2- 172.21.15.79:0/4188809225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84900ce7f0 0x7f84900cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f8496230700 1 --2- 172.21.15.79:0/4188809225 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8490159170 0x7f84900cd3c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.964+0000 7f8496230700 1 -- 172.21.15.79:0/4188809225 >> 172.21.15.79:0/4188809225 conn(0x7f849006d660 msgr2=0x7f849006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:28.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.965+0000 7f8496230700 1 -- 172.21.15.79:0/4188809225 shutdown_connections 2023-06-23T18:37:28.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.965+0000 7f8496230700 1 -- 172.21.15.79:0/4188809225 wait complete. 2023-06-23T18:37:28.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.966+0000 7f8496230700 1 Processor -- start 2023-06-23T18:37:28.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.966+0000 7f8496230700 1 -- start start 2023-06-23T18:37:28.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f8496230700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84900cd900 0x7f8490152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f8496230700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84900ce7f0 0x7f849015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f8496230700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8490159170 0x7f84900aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f8496230700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84900ae000 con 0x7f84900ce7f0 2023-06-23T18:37:28.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f8496230700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849015b9f0 con 0x7f84900cd900 2023-06-23T18:37:28.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f8496230700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84900ad660 con 0x7f8490159170 2023-06-23T18:37:28.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f848effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84900ce7f0 0x7f849015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f848effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84900ce7f0 0x7f849015afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42514/0 (socket says 172.21.15.79:42514) 2023-06-23T18:37:28.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f848effd700 1 -- 172.21.15.79:0/2248911764 learned_addr learned my addr 172.21.15.79:0/2248911764 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:28.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f848f7fe700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84900cd900 0x7f8490152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f848ffff700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8490159170 0x7f84900aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f848effd700 1 -- 172.21.15.79:0/2248911764 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8490159170 msgr2=0x7f84900aff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f848effd700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8490159170 0x7f84900aff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f848effd700 1 -- 172.21.15.79:0/2248911764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84900cd900 msgr2=0x7f8490152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:28.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f848effd700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84900cd900 0x7f8490152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:28.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f848effd700 1 -- 172.21.15.79:0/2248911764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847800b7f0 con 0x7f84900ce7f0 2023-06-23T18:37:28.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f848ffff700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8490159170 0x7f84900aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:28.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f848effd700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84900ce7f0 0x7f849015afe0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f84800011e0 tx=0x7f848000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:28.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f8487fff700 1 -- 172.21.15.79:0/2248911764 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480004c80 con 0x7f84900ce7f0 2023-06-23T18:37:28.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f8496230700 1 -- 172.21.15.79:0/2248911764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84900ad940 con 0x7f84900ce7f0 2023-06-23T18:37:28.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f8496230700 1 -- 172.21.15.79:0/2248911764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8490152db0 con 0x7f84900ce7f0 2023-06-23T18:37:28.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f8487fff700 1 -- 172.21.15.79:0/2248911764 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8480004de0 con 0x7f84900ce7f0 2023-06-23T18:37:28.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.967+0000 7f8487fff700 1 -- 172.21.15.79:0/2248911764 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800072d0 con 0x7f84900ce7f0 2023-06-23T18:37:28.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.968+0000 7f8496230700 1 -- 172.21.15.79:0/2248911764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84900ae140 con 0x7f84900ce7f0 2023-06-23T18:37:28.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.969+0000 7f8487fff700 1 -- 172.21.15.79:0/2248911764 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8480007430 con 0x7f84900ce7f0 2023-06-23T18:37:28.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.969+0000 7f8487fff700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f847006ace0 0x7f847006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:28.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.969+0000 7f8487fff700 1 -- 172.21.15.79:0/2248911764 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8480088d90 con 0x7f84900ce7f0 2023-06-23T18:37:28.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.969+0000 7f848f7fe700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f847006ace0 0x7f847006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:28.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.969+0000 7f848f7fe700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f847006ace0 0x7f847006d190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8478000c00 tx=0x7f847801c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:28.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:28.973+0000 7f8487fff700 1 -- 172.21.15.79:0/2248911764 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f84900ae140 con 0x7f84900ce7f0 2023-06-23T18:37:29.023 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:37:29.025 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:37:29.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.167+0000 7f8496230700 1 -- 172.21.15.79:0/2248911764 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f849015d840 con 0x7f847006ace0 2023-06-23T18:37:29.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.168+0000 7f8487fff700 1 -- 172.21.15.79:0/2248911764 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8478006d30 con 0x7f847006ace0 2023-06-23T18:37:29.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.170+0000 7f8496230700 1 -- 172.21.15.79:0/2248911764 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f847006ace0 msgr2=0x7f847006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:29.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.170+0000 7f8496230700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f847006ace0 0x7f847006d190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8478000c00 tx=0x7f847801c040 comp rx=0 tx=0).stop 2023-06-23T18:37:29.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.170+0000 7f8496230700 1 -- 172.21.15.79:0/2248911764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84900ce7f0 msgr2=0x7f849015afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:29.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.170+0000 7f8496230700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84900ce7f0 0x7f849015afe0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f84800011e0 tx=0x7f848000ca30 comp rx=0 tx=0).stop 2023-06-23T18:37:29.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.170+0000 7f8496230700 1 -- 172.21.15.79:0/2248911764 shutdown_connections 2023-06-23T18:37:29.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.170+0000 7f8496230700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8490159170 0x7f84900aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:29.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.170+0000 7f8496230700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f847006ace0 0x7f847006d190 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:29.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.170+0000 7f8496230700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84900cd900 0x7f8490152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:29.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.170+0000 7f8496230700 1 --2- 172.21.15.79:0/2248911764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84900ce7f0 0x7f849015afe0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:29.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.170+0000 7f8496230700 1 -- 172.21.15.79:0/2248911764 >> 172.21.15.79:0/2248911764 conn(0x7f849006d660 msgr2=0x7f84900af230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:29.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.170+0000 7f8496230700 1 -- 172.21.15.79:0/2248911764 shutdown_connections 2023-06-23T18:37:29.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.170+0000 7f8496230700 1 -- 172.21.15.79:0/2248911764 wait complete. 2023-06-23T18:37:29.178 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 7 to deep-scrub 2023-06-23T18:37:29.187 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:37:29.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.302+0000 7f17d23b4700 1 Processor -- start 2023-06-23T18:37:29.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.302+0000 7f17d23b4700 1 -- start start 2023-06-23T18:37:29.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.302+0000 7f17d23b4700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f17cc071490 0x7f17cc071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:29.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.302+0000 7f17d23b4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17cc0aee30 0x7f17cc071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:29.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.302+0000 7f17d23b4700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17cc1608f0 con 0x7f17cc0b1270 2023-06-23T18:37:29.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.302+0000 7f17d23b4700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17cc0722e0 con 0x7f17cc071490 2023-06-23T18:37:29.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.302+0000 7f17d23b4700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc160e20 con 0x7f17cc0aee30 2023-06-23T18:37:29.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.303+0000 7f17d0951700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17cc0aee30 0x7f17cc071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:29.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.303+0000 7f17cbfff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f17cc071490 0x7f17cc071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:29.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.303+0000 7f17d0951700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17cc0aee30 0x7f17cc071da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.79:35294/0 (socket says 172.21.15.79:35294) 2023-06-23T18:37:29.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.303+0000 7f17d0951700 1 -- 172.21.15.79:0/4249927340 learned_addr learned my addr 172.21.15.79:0/4249927340 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:29.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.303+0000 7f17cbfff700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f17cc071490 0x7f17cc071860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.79:48746/0 (socket says 172.21.15.79:48746) 2023-06-23T18:37:29.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.303+0000 7f17cb7fe700 1 --1- 172.21.15.79:0/4249927340 >> v1:172.21.15.155:6789/0 conn(0x7f17cc0b1270 0x7f17cc069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:53830/0 (socket says 172.21.15.79:53830) 2023-06-23T18:37:29.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.303+0000 7f17d0951700 1 -- 172.21.15.79:0/4249927340 >> v2:172.21.15.155:3300/0 conn(0x7f17cc071490 msgr2=0x7f17cc071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:29.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.303+0000 7f17d0951700 1 --2- 172.21.15.79:0/4249927340 >> v2:172.21.15.155:3300/0 conn(0x7f17cc071490 0x7f17cc071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:29.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.303+0000 7f17d0951700 1 -- 172.21.15.79:0/4249927340 >> v1:172.21.15.155:6789/0 conn(0x7f17cc0b1270 legacy=0x7f17cc069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:29.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.303+0000 7f17d0951700 1 -- 172.21.15.79:0/4249927340 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc0cd4f0 con 0x7f17cc0aee30 2023-06-23T18:37:29.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.303+0000 7f17cbfff700 1 --2- 172.21.15.79:0/4249927340 >> v2:172.21.15.155:3300/0 conn(0x7f17cc071490 0x7f17cc071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:29.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.303+0000 7f17d0951700 1 --2- 172.21.15.79:0/4249927340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17cc0aee30 0x7f17cc071da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009e50 tx=0x7f17c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56e5f7e54c557893 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:29.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.303+0000 7f17caffd700 1 -- 172.21.15.79:0/4249927340 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c400e070 con 0x7f17cc0aee30 2023-06-23T18:37:29.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.304+0000 7f17caffd700 1 -- 172.21.15.79:0/4249927340 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c400b910 con 0x7f17cc0aee30 2023-06-23T18:37:29.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.304+0000 7f17caffd700 1 -- 172.21.15.79:0/4249927340 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c400bcc0 con 0x7f17cc0aee30 2023-06-23T18:37:29.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.304+0000 7f17d23b4700 1 -- 172.21.15.79:0/4249927340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17cc0aee30 msgr2=0x7f17cc071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:29.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.304+0000 7f17d23b4700 1 --2- 172.21.15.79:0/4249927340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17cc0aee30 0x7f17cc071da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009e50 tx=0x7f17c400b330 comp rx=0 tx=0).stop 2023-06-23T18:37:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.304+0000 7f17d23b4700 1 -- 172.21.15.79:0/4249927340 shutdown_connections 2023-06-23T18:37:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.304+0000 7f17d23b4700 1 --2- 172.21.15.79:0/4249927340 >> v2:172.21.15.155:3300/0 conn(0x7f17cc071490 0x7f17cc071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.304+0000 7f17d23b4700 1 --2- 172.21.15.79:0/4249927340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17cc0aee30 0x7f17cc071da0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:29.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.304+0000 7f17d23b4700 1 -- 172.21.15.79:0/4249927340 >> 172.21.15.79:0/4249927340 conn(0x7f17cc06d660 msgr2=0x7f17cc06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:29.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.304+0000 7f17d23b4700 1 -- 172.21.15.79:0/4249927340 shutdown_connections 2023-06-23T18:37:29.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.304+0000 7f17d23b4700 1 -- 172.21.15.79:0/4249927340 wait complete. 2023-06-23T18:37:29.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.305+0000 7f17d23b4700 1 Processor -- start 2023-06-23T18:37:29.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.305+0000 7f17d23b4700 1 -- start start 2023-06-23T18:37:29.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.306+0000 7f17d23b4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17cc071490 0x7f17cc0b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:29.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.306+0000 7f17d23b4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17cc0aee30 0x7f17cc0b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:29.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.306+0000 7f17d23b4700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17cc0b1270 0x7f17cc0b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:29.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.306+0000 7f17d23b4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0c70c0 con 0x7f17cc0aee30 2023-06-23T18:37:29.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.306+0000 7f17d23b4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0c7310 con 0x7f17cc071490 2023-06-23T18:37:29.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.306+0000 7f17d23b4700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0c75f0 con 0x7f17cc0b1270 2023-06-23T18:37:29.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.306+0000 7f17cb7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17cc0aee30 0x7f17cc0b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:29.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.306+0000 7f17cb7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17cc0aee30 0x7f17cc0b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42526/0 (socket says 172.21.15.79:42526) 2023-06-23T18:37:29.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.306+0000 7f17cb7fe700 1 -- 172.21.15.79:0/2102207530 learned_addr learned my addr 172.21.15.79:0/2102207530 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:29.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.306+0000 7f17d0951700 1 --2- 172.21.15.79:0/2102207530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17cc0b1270 0x7f17cc0b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:29.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.306+0000 7f17cbfff700 1 --2- 172.21.15.79:0/2102207530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17cc071490 0x7f17cc0b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:29.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.307+0000 7f17cb7fe700 1 -- 172.21.15.79:0/2102207530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17cc0b1270 msgr2=0x7f17cc0b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:29.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.307+0000 7f17cb7fe700 1 --2- 172.21.15.79:0/2102207530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17cc0b1270 0x7f17cc0b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:29.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.307+0000 7f17cb7fe700 1 -- 172.21.15.79:0/2102207530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17cc071490 msgr2=0x7f17cc0b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:29.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.307+0000 7f17cb7fe700 1 --2- 172.21.15.79:0/2102207530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17cc071490 0x7f17cc0b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.307+0000 7f17cb7fe700 1 -- 172.21.15.79:0/2102207530 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc0094a0 con 0x7f17cc0aee30 2023-06-23T18:37:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.307+0000 7f17cb7fe700 1 --2- 172.21.15.79:0/2102207530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17cc0aee30 0x7f17cc0b2b60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c500 tx=0x7f17c000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.307+0000 7f17c8ff9700 1 -- 172.21.15.79:0/2102207530 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0005cb0 con 0x7f17cc0aee30 2023-06-23T18:37:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.307+0000 7f17d23b4700 1 -- 172.21.15.79:0/2102207530 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c4009b30 con 0x7f17cc0aee30 2023-06-23T18:37:29.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.307+0000 7f17c8ff9700 1 -- 172.21.15.79:0/2102207530 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c0005e10 con 0x7f17cc0aee30 2023-06-23T18:37:29.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.307+0000 7f17c8ff9700 1 -- 172.21.15.79:0/2102207530 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c00072f0 con 0x7f17cc0aee30 2023-06-23T18:37:29.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.307+0000 7f17d23b4700 1 -- 172.21.15.79:0/2102207530 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc0c7bb0 con 0x7f17cc0aee30 2023-06-23T18:37:29.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.308+0000 7f17d23b4700 1 -- 172.21.15.79:0/2102207530 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17cc0ce300 con 0x7f17cc0aee30 2023-06-23T18:37:29.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.308+0000 7f17c8ff9700 1 -- 172.21.15.79:0/2102207530 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f17c0007450 con 0x7f17cc0aee30 2023-06-23T18:37:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.309+0000 7f17c8ff9700 1 --2- 172.21.15.79:0/2102207530 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f17b806ace0 0x7f17b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.309+0000 7f17c8ff9700 1 -- 172.21.15.79:0/2102207530 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f17c00884e0 con 0x7f17cc0aee30 2023-06-23T18:37:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.309+0000 7f17cbfff700 1 --2- 172.21.15.79:0/2102207530 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f17b806ace0 0x7f17b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.309+0000 7f17cbfff700 1 --2- 172.21.15.79:0/2102207530 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f17b806ace0 0x7f17b806d190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009cd0 tx=0x7f17bc009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:29.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.313+0000 7f17c8ff9700 1 -- 172.21.15.79:0/2102207530 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f17c00538a0 con 0x7f17cc0aee30 2023-06-23T18:37:29.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.498+0000 7f17d23b4700 1 -- 172.21.15.79:0/2102207530 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f17cc0cd790 con 0x7f17b806ace0 2023-06-23T18:37:29.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.501+0000 7f17c8ff9700 1 -- 172.21.15.79:0/2102207530 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72794 (secure 0 0 0) 0x7f17bc009e10 con 0x7f17b806ace0 2023-06-23T18:37:29.507 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:29.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.503+0000 7f17d23b4700 1 -- 172.21.15.79:0/2102207530 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f17b806ace0 msgr2=0x7f17b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:29.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.503+0000 7f17d23b4700 1 --2- 172.21.15.79:0/2102207530 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f17b806ace0 0x7f17b806d190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009cd0 tx=0x7f17bc009210 comp rx=0 tx=0).stop 2023-06-23T18:37:29.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.503+0000 7f17d23b4700 1 -- 172.21.15.79:0/2102207530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17cc0aee30 msgr2=0x7f17cc0b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:29.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.503+0000 7f17d23b4700 1 --2- 172.21.15.79:0/2102207530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17cc0aee30 0x7f17cc0b2b60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c500 tx=0x7f17c000c810 comp rx=0 tx=0).stop 2023-06-23T18:37:29.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.503+0000 7f17d23b4700 1 -- 172.21.15.79:0/2102207530 shutdown_connections 2023-06-23T18:37:29.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.503+0000 7f17d23b4700 1 --2- 172.21.15.79:0/2102207530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17cc0b1270 0x7f17cc0b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:29.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.503+0000 7f17d23b4700 1 --2- 172.21.15.79:0/2102207530 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f17b806ace0 0x7f17b806d190 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:29.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.503+0000 7f17d23b4700 1 --2- 172.21.15.79:0/2102207530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17cc071490 0x7f17cc0b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:29.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.503+0000 7f17d23b4700 1 --2- 172.21.15.79:0/2102207530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17cc0aee30 0x7f17cc0b2b60 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:29.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.503+0000 7f17d23b4700 1 -- 172.21.15.79:0/2102207530 >> 172.21.15.79:0/2102207530 conn(0x7f17cc06d660 msgr2=0x7f17cc0cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:29.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.503+0000 7f17d23b4700 1 -- 172.21.15.79:0/2102207530 shutdown_connections 2023-06-23T18:37:29.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:29.503+0000 7f17d23b4700 1 -- 172.21.15.79:0/2102207530 wait complete. 2023-06-23T18:37:29.511 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-23T18:37:29.520 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2023-06-23T18:37:28.714407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220124,"kb_used_data":4508,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754596,"statfs":{"total":773094113280,"available":772868706304,"internally_reserved":0,"allocated":4616192,"data_stored":2608053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.729104"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.996008+0000","last_change":"2023-06-23T18:35:48.920009+0000","last_active":"2023-06-23T18:37:20.996008+0000","last_peered":"2023-06-23T18:37:20.996008+0000","last_clean":"2023-06-23T18:37:20.996008+0000","last_became_active":"2023-06-23T18:35:48.919686+0000","last_became_peered":"2023-06-23T18:35:48.919686+0000","last_unstale":"2023-06-23T18:37:20.996008+0000","last_undegraded":"2023-06-23T18:37:20.996008+0000","last_fullsized":"2023-06-23T18:37:20.996008+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:21:52.892529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:21.401877+0000","last_change":"2023-06-23T18:35:47.962164+0000","last_active":"2023-06-23T18:37:21.401877+0000","last_peered":"2023-06-23T18:37:21.401877+0000","last_clean":"2023-06-23T18:37:21.401877+0000","last_became_active":"2023-06-23T18:35:47.961790+0000","last_became_peered":"2023-06-23T18:35:47.961790+0000","last_unstale":"2023-06-23T18:37:21.401877+0000","last_undegraded":"2023-06-23T18:37:21.401877+0000","last_fullsized":"2023-06-23T18:37:21.401877+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:57:42.414472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:21.404998+0000","last_change":"2023-06-23T18:35:47.965867+0000","last_active":"2023-06-23T18:37:21.404998+0000","last_peered":"2023-06-23T18:37:21.404998+0000","last_clean":"2023-06-23T18:37:21.404998+0000","last_became_active":"2023-06-23T18:35:47.965580+0000","last_became_peered":"2023-06-23T18:35:47.965580+0000","last_unstale":"2023-06-23T18:37:21.404998+0000","last_undegraded":"2023-06-23T18:37:21.404998+0000","last_fullsized":"2023-06-23T18:37:21.404998+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:20:11.367194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":199,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:21.403583+0000","last_change":"2023-06-23T18:35:47.968534+0000","last_active":"2023-06-23T18:37:21.403583+0000","last_peered":"2023-06-23T18:37:21.403583+0000","last_clean":"2023-06-23T18:37:21.403583+0000","last_became_active":"2023-06-23T18:35:47.967508+0000","last_became_peered":"2023-06-23T18:35:47.967508+0000","last_unstale":"2023-06-23T18:37:21.403583+0000","last_undegraded":"2023-06-23T18:37:21.403583+0000","last_fullsized":"2023-06-23T18:37:21.403583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:10:53.001920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:21.403370+0000","last_change":"2023-06-23T18:35:48.736407+0000","last_active":"2023-06-23T18:37:21.403370+0000","last_peered":"2023-06-23T18:37:21.403370+0000","last_clean":"2023-06-23T18:37:21.403370+0000","last_became_active":"2023-06-23T18:35:48.735621+0000","last_became_peered":"2023-06-23T18:35:48.735621+0000","last_unstale":"2023-06-23T18:37:21.403370+0000","last_undegraded":"2023-06-23T18:37:21.403370+0000","last_fullsized":"2023-06-23T18:37:21.403370+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:16:41.861826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:21.403422+0000","last_change":"2023-06-23T18:35:48.736449+0000","last_active":"2023-06-23T18:37:21.403422+0000","last_peered":"2023-06-23T18:37:21.403422+0000","last_clean":"2023-06-23T18:37:21.403422+0000","last_became_active":"2023-06-23T18:35:48.735664+0000","last_became_peered":"2023-06-23T18:35:48.735664+0000","last_unstale":"2023-06-23T18:37:21.403422+0000","last_undegraded":"2023-06-23T18:37:21.403422+0000","last_fullsized":"2023-06-23T18:37:21.403422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:42:53.202060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:20.995032+0000","last_change":"2023-06-23T18:35:48.735713+0000","last_active":"2023-06-23T18:37:20.995032+0000","last_peered":"2023-06-23T18:37:20.995032+0000","last_clean":"2023-06-23T18:37:20.995032+0000","last_became_active":"2023-06-23T18:35:48.735236+0000","last_became_peered":"2023-06-23T18:35:48.735236+0000","last_unstale":"2023-06-23T18:37:20.995032+0000","last_undegraded":"2023-06-23T18:37:20.995032+0000","last_fullsized":"2023-06-23T18:37:20.995032+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:44:34.986048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:21.402233+0000","last_change":"2023-06-23T18:35:48.735631+0000","last_active":"2023-06-23T18:37:21.402233+0000","last_peered":"2023-06-23T18:37:21.402233+0000","last_clean":"2023-06-23T18:37:21.402233+0000","last_became_active":"2023-06-23T18:35:48.735255+0000","last_became_peered":"2023-06-23T18:35:48.735255+0000","last_unstale":"2023-06-23T18:37:21.402233+0000","last_undegraded":"2023-06-23T18:37:21.402233+0000","last_fullsized":"2023-06-23T18:37:21.402233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:34:28.376470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:21.402502+0000","last_change":"2023-06-23T18:35:47.970359+0000","last_active":"2023-06-23T18:37:21.402502+0000","last_peered":"2023-06-23T18:37:21.402502+0000","last_clean":"2023-06-23T18:37:21.402502+0000","last_became_active":"2023-06-23T18:35:47.970118+0000","last_became_peered":"2023-06-23T18:35:47.970118+0000","last_unstale":"2023-06-23T18:37:21.402502+0000","last_undegraded":"2023-06-23T18:37:21.402502+0000","last_fullsized":"2023-06-23T18:37:21.402502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:35:46.952958+0000","last_clean_scrub_stamp":"2023-06-23T18:35:46.952958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:59:14.078142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":1019904,"data_stored":767932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.30499999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":11.071,"5min":11.071,"15min":11.071},"last":0.48199999999999998}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":11.118,"5min":11.118,"15min":11.118},"last":0.40600000000000003},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":11.103999999999999,"5min":11.103999999999999,"15min":11.103999999999999},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.498}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":0.53300000000000003}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.59299999999999997}]},{"osd":5,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56799999999999995}]},{"osd":6,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.622},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":178925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.505}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.623},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.53500000000000003}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.65400000000000003}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.63600000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.39500000000000002}]},{"osd":5,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.376}]},{"osd":7,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.48399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":1024000,"data_stored":769293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.8680000000000003,"5min":7.8680000000000003,"15min":7.8680000000000003},"last":0.251}]},{"osd":2,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.6429999999999998,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":0.377},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":9.4209999999999994,"5min":9.4209999999999994,"15min":9.4209999999999994},"last":0.39100000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":5.6790000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":7.9039999999999999,"5min":7.9039999999999999,"15min":7.9039999999999999},"last":0.40500000000000003}]},{"osd":4,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":11.385999999999999,"5min":11.385999999999999,"15min":11.385999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":7.9649999999999999,"5min":7.9649999999999999,"15min":7.9649999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":11.259,"5min":11.259,"15min":11.259},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":7.9909999999999997,"5min":7.9909999999999997,"15min":7.9909999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":11.346,"5min":11.346,"15min":11.346},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":7.9359999999999999,"5min":7.9359999999999999,"15min":7.9359999999999999},"last":0.73799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":177564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":16.004999999999999,"5min":16.004999999999999,"15min":16.004999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":16.073,"5min":16.073,"15min":16.073},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":8.2460000000000004,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":8.2129999999999992,"5min":8.2129999999999992,"15min":8.2129999999999992},"last":0.42899999999999999}]},{"osd":3,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":16.036999999999999,"5min":16.036999999999999,"15min":16.036999999999999},"last":0.55500000000000005}]},{"osd":4,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":8.0600000000000005,"5min":8.0600000000000005,"15min":8.0600000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":8.2739999999999991,"5min":8.2739999999999991,"15min":8.2739999999999991},"last":0.82399999999999995}]},{"osd":5,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":8.3320000000000007,"5min":8.3320000000000007,"15min":8.3320000000000007},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":8.0299999999999994,"5min":8.0299999999999994,"15min":8.0299999999999994},"last":0.88600000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.3030000000000008,"5min":8.3030000000000008,"15min":8.3030000000000008},"last":0.873},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":8.3800000000000008,"5min":8.3800000000000008,"15min":8.3800000000000008},"last":0.66400000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":8.3580000000000005,"5min":8.3580000000000005,"15min":8.3580000000000005},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":8.0839999999999996,"5min":8.0839999999999996,"15min":8.0839999999999996},"last":0.84099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":177564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47499999999999998}]},{"osd":1,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.41199999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":5.0899999999999999,"5min":5.0899999999999999,"15min":5.0899999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.89100000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.871},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.73899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":178925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":4.359,"5min":4.359,"15min":4.359},"last":0.29799999999999999}]},{"osd":1,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.245},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.313}]},{"osd":2,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.34799999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.623}]},{"osd":5,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56999999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":178925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.57999999999999996}]},{"osd":1,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.496},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.623}]},{"osd":3,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.442},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.64300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":178925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60299999999999998}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.63}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.65800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:37:29.525 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-06-23T18:35:46.952958+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=35, tm_sec=46, tm_wday=4, tm_yday=174, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=37, tm_sec=24, tm_wday=4, tm_yday=174, tm_isdst=0) 2023-06-23T18:37:29.526 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-06-23T18:35:46.952958+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=35, tm_sec=46, tm_wday=4, tm_yday=174, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=37, tm_sec=24, tm_wday=4, tm_yday=174, tm_isdst=0) 2023-06-23T18:37:29.526 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-06-23T18:35:46.952958+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=35, tm_sec=46, tm_wday=4, tm_yday=174, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=37, tm_sec=24, tm_wday=4, tm_yday=174, tm_isdst=0) 2023-06-23T18:37:29.527 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-23T18:35:46.952958+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=35, tm_sec=46, tm_wday=4, tm_yday=174, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=37, tm_sec=24, tm_wday=4, tm_yday=174, tm_isdst=0) 2023-06-23T18:37:29.527 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-06-23T18:35:46.952958+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=35, tm_sec=46, tm_wday=4, tm_yday=174, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=37, tm_sec=24, tm_wday=4, tm_yday=174, tm_isdst=0) 2023-06-23T18:37:29.528 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-06-23T18:35:46.952958+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=35, tm_sec=46, tm_wday=4, tm_yday=174, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=37, tm_sec=24, tm_wday=4, tm_yday=174, tm_isdst=0) 2023-06-23T18:37:29.528 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-06-23T18:35:46.952958+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=35, tm_sec=46, tm_wday=4, tm_yday=174, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=37, tm_sec=24, tm_wday=4, tm_yday=174, tm_isdst=0) 2023-06-23T18:37:29.529 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-06-23T18:35:46.952958+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=35, tm_sec=46, tm_wday=4, tm_yday=174, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=37, tm_sec=24, tm_wday=4, tm_yday=174, tm_isdst=0) 2023-06-23T18:37:29.530 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-06-23T18:35:46.952958+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=35, tm_sec=46, tm_wday=4, tm_yday=174, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=23, tm_hour=18, tm_min=37, tm_sec=24, tm_wday=4, tm_yday=174, tm_isdst=0) 2023-06-23T18:37:29.530 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-23T18:37:49.531 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:37:49.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.663+0000 7f62fa160700 1 Processor -- start 2023-06-23T18:37:49.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.663+0000 7f62fa160700 1 -- start start 2023-06-23T18:37:49.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.664+0000 7f62fa160700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f62f4159130 0x7f62f40c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:49.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.664+0000 7f62fa160700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62f40c5000 0x7f62f4070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:49.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.664+0000 7f62fa160700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62f415d930 con 0x7f62f4070770 2023-06-23T18:37:49.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.664+0000 7f62fa160700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f415cd20 con 0x7f62f4159130 2023-06-23T18:37:49.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.664+0000 7f62fa160700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f415d000 con 0x7f62f40c5000 2023-06-23T18:37:49.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.664+0000 7f62f37fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f62f4159130 0x7f62f40c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:49.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.664+0000 7f62f37fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f62f4159130 0x7f62f40c4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42534/0 (socket says 172.21.15.79:42534) 2023-06-23T18:37:49.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.664+0000 7f62f37fe700 1 -- 172.21.15.79:0/1715370106 learned_addr learned my addr 172.21.15.79:0/1715370106 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:49.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.664+0000 7f62f37fe700 1 -- 172.21.15.79:0/1715370106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62f40c5000 msgr2=0x7f62f4070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:49.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.664+0000 7f62f2ffd700 1 --2- 172.21.15.79:0/1715370106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62f40c5000 0x7f62f4070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:49.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.665+0000 7f62f37fe700 1 --2- 172.21.15.79:0/1715370106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62f40c5000 0x7f62f4070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:49.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.665+0000 7f62f37fe700 1 -- 172.21.15.79:0/1715370106 >> v1:172.21.15.155:6789/0 conn(0x7f62f4070770 legacy=0x7f62f40cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:49.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.665+0000 7f62f37fe700 1 -- 172.21.15.79:0/1715370106 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f4070bd0 con 0x7f62f4159130 2023-06-23T18:37:49.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.665+0000 7f62f2ffd700 1 --2- 172.21.15.79:0/1715370106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62f40c5000 0x7f62f4070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:49.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.665+0000 7f62f37fe700 1 --2- 172.21.15.79:0/1715370106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62f4159130 0x7f62f40c4ac0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f62e80098f0 tx=0x7f62e8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e74e1383d0a7bbb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:49.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.665+0000 7f62f27fc700 1 -- 172.21.15.79:0/1715370106 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e8004230 con 0x7f62f4159130 2023-06-23T18:37:49.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.665+0000 7f62f27fc700 1 -- 172.21.15.79:0/1715370106 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e8004390 con 0x7f62f4159130 2023-06-23T18:37:49.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.665+0000 7f62f27fc700 1 -- 172.21.15.79:0/1715370106 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e80036a0 con 0x7f62f4159130 2023-06-23T18:37:49.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.665+0000 7f62fa160700 1 -- 172.21.15.79:0/1715370106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62f4159130 msgr2=0x7f62f40c4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:49.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.665+0000 7f62fa160700 1 --2- 172.21.15.79:0/1715370106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62f4159130 0x7f62f40c4ac0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f62e80098f0 tx=0x7f62e8009c00 comp rx=0 tx=0).stop 2023-06-23T18:37:49.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.665+0000 7f62fa160700 1 -- 172.21.15.79:0/1715370106 shutdown_connections 2023-06-23T18:37:49.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.665+0000 7f62fa160700 1 --2- 172.21.15.79:0/1715370106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62f4159130 0x7f62f40c4ac0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:49.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.665+0000 7f62fa160700 1 --2- 172.21.15.79:0/1715370106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62f40c5000 0x7f62f4070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:49.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.665+0000 7f62fa160700 1 -- 172.21.15.79:0/1715370106 >> 172.21.15.79:0/1715370106 conn(0x7f62f40cf3d0 msgr2=0x7f62f40cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:49.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.666+0000 7f62fa160700 1 -- 172.21.15.79:0/1715370106 shutdown_connections 2023-06-23T18:37:49.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.666+0000 7f62fa160700 1 -- 172.21.15.79:0/1715370106 wait complete. 2023-06-23T18:37:49.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.667+0000 7f62fa160700 1 Processor -- start 2023-06-23T18:37:49.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.667+0000 7f62fa160700 1 -- start start 2023-06-23T18:37:49.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62fa160700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62f4070770 0x7f62f414a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:49.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62fa160700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62f40c5000 0x7f62f40aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:49.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62fa160700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62f4159130 0x7f62f40af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:49.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62fa160700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f40b18a0 con 0x7f62f4159130 2023-06-23T18:37:49.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62fa160700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f41efd20 con 0x7f62f40c5000 2023-06-23T18:37:49.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62fa160700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f41ef890 con 0x7f62f4070770 2023-06-23T18:37:49.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62f4159130 0x7f62f40af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:49.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62f4159130 0x7f62f40af500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42544/0 (socket says 172.21.15.79:42544) 2023-06-23T18:37:49.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62f3fff700 1 -- 172.21.15.79:0/106050892 learned_addr learned my addr 172.21.15.79:0/106050892 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:49.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62f37fe700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62f4070770 0x7f62f414a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:49.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62f2ffd700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62f40c5000 0x7f62f40aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:49.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62f3fff700 1 -- 172.21.15.79:0/106050892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62f4070770 msgr2=0x7f62f414a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:49.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62f3fff700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62f4070770 0x7f62f414a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:49.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62f3fff700 1 -- 172.21.15.79:0/106050892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62f40c5000 msgr2=0x7f62f40aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:49.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62f3fff700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62f40c5000 0x7f62f40aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:49.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62f3fff700 1 -- 172.21.15.79:0/106050892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e8009550 con 0x7f62f4159130 2023-06-23T18:37:49.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62f37fe700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62f4070770 0x7f62f414a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:49.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62f3fff700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62f4159130 0x7f62f40af500 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f62ec00b970 tx=0x7f62ec00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:49.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62dbfff700 1 -- 172.21.15.79:0/106050892 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec00c9b0 con 0x7f62f4159130 2023-06-23T18:37:49.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62dbfff700 1 -- 172.21.15.79:0/106050892 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ec00cb10 con 0x7f62f4159130 2023-06-23T18:37:49.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.668+0000 7f62fa160700 1 -- 172.21.15.79:0/106050892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f41eed00 con 0x7f62f4159130 2023-06-23T18:37:49.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.669+0000 7f62dbfff700 1 -- 172.21.15.79:0/106050892 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec014440 con 0x7f62f4159130 2023-06-23T18:37:49.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.669+0000 7f62fa160700 1 -- 172.21.15.79:0/106050892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f414aa10 con 0x7f62f4159130 2023-06-23T18:37:49.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.669+0000 7f62fa160700 1 -- 172.21.15.79:0/106050892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f41ef9d0 con 0x7f62f4159130 2023-06-23T18:37:49.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.670+0000 7f62dbfff700 1 -- 172.21.15.79:0/106050892 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f62ec01d3b0 con 0x7f62f4159130 2023-06-23T18:37:49.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.670+0000 7f62dbfff700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f62e006ace0 0x7f62e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:49.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.670+0000 7f62dbfff700 1 -- 172.21.15.79:0/106050892 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f62ec088f10 con 0x7f62f4159130 2023-06-23T18:37:49.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.670+0000 7f62f37fe700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f62e006ace0 0x7f62e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:49.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.671+0000 7f62f37fe700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f62e006ace0 0x7f62e006d190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009c40 tx=0x7f62e8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:49.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.675+0000 7f62dbfff700 1 -- 172.21.15.79:0/106050892 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f62ec0542c0 con 0x7f62f4159130 2023-06-23T18:37:49.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.855+0000 7f62fa160700 1 -- 172.21.15.79:0/106050892 --> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62f415d5c0 con 0x7f62e006ace0 2023-06-23T18:37:49.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.857+0000 7f62dbfff700 1 -- 172.21.15.79:0/106050892 <== mgr.4103 v2:172.21.15.79:6832/1804789972 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72846 (secure 0 0 0) 0x7f62e8020070 con 0x7f62e006ace0 2023-06-23T18:37:49.863 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:49.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.859+0000 7f62fa160700 1 -- 172.21.15.79:0/106050892 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f62e006ace0 msgr2=0x7f62e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:49.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.859+0000 7f62fa160700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f62e006ace0 0x7f62e006d190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009c40 tx=0x7f62e8008030 comp rx=0 tx=0).stop 2023-06-23T18:37:49.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.859+0000 7f62fa160700 1 -- 172.21.15.79:0/106050892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62f4159130 msgr2=0x7f62f40af500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:49.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.859+0000 7f62fa160700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62f4159130 0x7f62f40af500 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f62ec00b970 tx=0x7f62ec00bd30 comp rx=0 tx=0).stop 2023-06-23T18:37:49.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.859+0000 7f62fa160700 1 -- 172.21.15.79:0/106050892 shutdown_connections 2023-06-23T18:37:49.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.859+0000 7f62fa160700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62f4159130 0x7f62f40af500 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:49.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.859+0000 7f62fa160700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62f4070770 0x7f62f414a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:49.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.859+0000 7f62fa160700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f62e006ace0 0x7f62e006d190 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:49.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.859+0000 7f62fa160700 1 --2- 172.21.15.79:0/106050892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62f40c5000 0x7f62f40aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:49.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.860+0000 7f62fa160700 1 -- 172.21.15.79:0/106050892 >> 172.21.15.79:0/106050892 conn(0x7f62f40cf3d0 msgr2=0x7f62f40b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:49.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.860+0000 7f62fa160700 1 -- 172.21.15.79:0/106050892 shutdown_connections 2023-06-23T18:37:49.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.860+0000 7f62fa160700 1 -- 172.21.15.79:0/106050892 wait complete. 2023-06-23T18:37:49.867 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-23T18:37:49.877 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2023-06-23T18:37:48.718646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220124,"kb_used_data":4508,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754596,"statfs":{"total":773094113280,"available":772868706304,"internally_reserved":0,"allocated":4616192,"data_stored":2608053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002652"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":60,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:29.070920+0000","last_change":"2023-06-23T18:37:29.070958+0000","last_active":"2023-06-23T18:37:29.070920+0000","last_peered":"2023-06-23T18:37:29.070920+0000","last_clean":"2023-06-23T18:37:29.070920+0000","last_became_active":"2023-06-23T18:35:48.919686+0000","last_became_peered":"2023-06-23T18:35:48.919686+0000","last_unstale":"2023-06-23T18:37:29.070920+0000","last_undegraded":"2023-06-23T18:37:29.070920+0000","last_fullsized":"2023-06-23T18:37:29.070920+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:37:29.070831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:37:29.070831+0000","last_clean_scrub_stamp":"2023-06-23T18:37:29.070831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:04:34.673144+0000","scrub_duration":0.001384873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:30.179440+0000","last_change":"2023-06-23T18:37:30.179484+0000","last_active":"2023-06-23T18:37:30.179440+0000","last_peered":"2023-06-23T18:37:30.179440+0000","last_clean":"2023-06-23T18:37:30.179440+0000","last_became_active":"2023-06-23T18:35:47.961790+0000","last_became_peered":"2023-06-23T18:35:47.961790+0000","last_unstale":"2023-06-23T18:37:30.179440+0000","last_undegraded":"2023-06-23T18:37:30.179440+0000","last_fullsized":"2023-06-23T18:37:30.179440+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:37:30.179338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:37:30.179338+0000","last_clean_scrub_stamp":"2023-06-23T18:37:30.179338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:52:33.621302+0000","scrub_duration":0.001676681,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":58,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:26.311028+0000","last_change":"2023-06-23T18:37:26.311073+0000","last_active":"2023-06-23T18:37:26.311028+0000","last_peered":"2023-06-23T18:37:26.311028+0000","last_clean":"2023-06-23T18:37:26.311028+0000","last_became_active":"2023-06-23T18:35:47.965580+0000","last_became_peered":"2023-06-23T18:35:47.965580+0000","last_unstale":"2023-06-23T18:37:26.311028+0000","last_undegraded":"2023-06-23T18:37:26.311028+0000","last_fullsized":"2023-06-23T18:37:26.311028+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:37:26.310943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:37:26.310943+0000","last_clean_scrub_stamp":"2023-06-23T18:37:26.310943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:08:12.931803+0000","scrub_duration":0.001925923,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"37'119","reported_seq":210,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:39.698011+0000","last_change":"2023-06-23T18:37:39.698070+0000","last_active":"2023-06-23T18:37:39.698011+0000","last_peered":"2023-06-23T18:37:39.698011+0000","last_clean":"2023-06-23T18:37:39.698011+0000","last_became_active":"2023-06-23T18:35:47.967508+0000","last_became_peered":"2023-06-23T18:35:47.967508+0000","last_unstale":"2023-06-23T18:37:39.698011+0000","last_undegraded":"2023-06-23T18:37:39.698011+0000","last_fullsized":"2023-06-23T18:37:39.698011+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'119","last_scrub_stamp":"2023-06-23T18:37:39.697914+0000","last_deep_scrub":"37'119","last_deep_scrub_stamp":"2023-06-23T18:37:39.697914+0000","last_clean_scrub_stamp":"2023-06-23T18:37:39.697914+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:24:58.313727+0000","scrub_duration":0.0079600469999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":58,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:33.583116+0000","last_change":"2023-06-23T18:37:33.583164+0000","last_active":"2023-06-23T18:37:33.583116+0000","last_peered":"2023-06-23T18:37:33.583116+0000","last_clean":"2023-06-23T18:37:33.583116+0000","last_became_active":"2023-06-23T18:35:48.735621+0000","last_became_peered":"2023-06-23T18:35:48.735621+0000","last_unstale":"2023-06-23T18:37:33.583116+0000","last_undegraded":"2023-06-23T18:37:33.583116+0000","last_fullsized":"2023-06-23T18:37:33.583116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:37:33.582980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:37:33.582980+0000","last_clean_scrub_stamp":"2023-06-23T18:37:33.582980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:52:56.624861+0000","scrub_duration":0.0019637600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":58,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:29.542004+0000","last_change":"2023-06-23T18:37:29.542050+0000","last_active":"2023-06-23T18:37:29.542004+0000","last_peered":"2023-06-23T18:37:29.542004+0000","last_clean":"2023-06-23T18:37:29.542004+0000","last_became_active":"2023-06-23T18:35:48.735664+0000","last_became_peered":"2023-06-23T18:35:48.735664+0000","last_unstale":"2023-06-23T18:37:29.542004+0000","last_undegraded":"2023-06-23T18:37:29.542004+0000","last_fullsized":"2023-06-23T18:37:29.542004+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:37:29.541891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:37:29.541891+0000","last_clean_scrub_stamp":"2023-06-23T18:37:29.541891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:38:30.979404+0000","scrub_duration":0.001735739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":60,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:29.627903+0000","last_change":"2023-06-23T18:37:29.627963+0000","last_active":"2023-06-23T18:37:29.627903+0000","last_peered":"2023-06-23T18:37:29.627903+0000","last_clean":"2023-06-23T18:37:29.627903+0000","last_became_active":"2023-06-23T18:35:48.735236+0000","last_became_peered":"2023-06-23T18:35:48.735236+0000","last_unstale":"2023-06-23T18:37:29.627903+0000","last_undegraded":"2023-06-23T18:37:29.627903+0000","last_fullsized":"2023-06-23T18:37:29.627903+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:37:29.627785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:37:29.627785+0000","last_clean_scrub_stamp":"2023-06-23T18:37:29.627785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:38:38.914460+0000","scrub_duration":0.001724894,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":58,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:29.676895+0000","last_change":"2023-06-23T18:37:29.676952+0000","last_active":"2023-06-23T18:37:29.676895+0000","last_peered":"2023-06-23T18:37:29.676895+0000","last_clean":"2023-06-23T18:37:29.676895+0000","last_became_active":"2023-06-23T18:35:48.735255+0000","last_became_peered":"2023-06-23T18:35:48.735255+0000","last_unstale":"2023-06-23T18:37:29.676895+0000","last_undegraded":"2023-06-23T18:37:29.676895+0000","last_fullsized":"2023-06-23T18:37:29.676895+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:37:29.676773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:37:29.676773+0000","last_clean_scrub_stamp":"2023-06-23T18:37:29.676773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:28:39.900559+0000","scrub_duration":0.001828564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":58,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-06-23T18:37:33.686977+0000","last_change":"2023-06-23T18:37:33.687027+0000","last_active":"2023-06-23T18:37:33.686977+0000","last_peered":"2023-06-23T18:37:33.686977+0000","last_clean":"2023-06-23T18:37:33.686977+0000","last_became_active":"2023-06-23T18:35:47.970118+0000","last_became_peered":"2023-06-23T18:35:47.970118+0000","last_unstale":"2023-06-23T18:37:33.686977+0000","last_undegraded":"2023-06-23T18:37:33.686977+0000","last_fullsized":"2023-06-23T18:37:33.686977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:37:33.686855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:37:33.686855+0000","last_clean_scrub_stamp":"2023-06-23T18:37:33.686855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:01:12.386038+0000","scrub_duration":0.0018919410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":1019904,"data_stored":767932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":11.071,"5min":11.071,"15min":11.071},"last":0.59399999999999997}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":11.118,"5min":11.118,"15min":11.118},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":11.103999999999999,"5min":11.103999999999999,"15min":11.103999999999999},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.57499999999999996}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":0.49399999999999999}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.41899999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.47699999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.63200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":178925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.495},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.55600000000000005}]},{"osd":5,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53300000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.504},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.51500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":1024000,"data_stored":769293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.374},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.8680000000000003,"5min":7.8680000000000003,"15min":7.8680000000000003},"last":0.495}]},{"osd":2,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.6429999999999998,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":9.4209999999999994,"5min":9.4209999999999994,"15min":9.4209999999999994},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":5.6790000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":7.9039999999999999,"5min":7.9039999999999999,"15min":7.9039999999999999},"last":0.40899999999999997}]},{"osd":4,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":11.385999999999999,"5min":11.385999999999999,"15min":11.385999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58099999999999996}]},{"osd":5,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":7.9649999999999999,"5min":7.9649999999999999,"15min":7.9649999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":11.259,"5min":11.259,"15min":11.259},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":7.9909999999999997,"5min":7.9909999999999997,"15min":7.9909999999999997},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":11.346,"5min":11.346,"15min":11.346},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":7.9359999999999999,"5min":7.9359999999999999,"15min":7.9359999999999999},"last":0.67200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":177564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":16.004999999999999,"5min":16.004999999999999,"15min":16.004999999999999},"last":0.307},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":16.073,"5min":16.073,"15min":16.073},"last":0.371}]},{"osd":2,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":8.2460000000000004,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":8.2129999999999992,"5min":8.2129999999999992,"15min":8.2129999999999992},"last":0.42399999999999999}]},{"osd":3,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":16.036999999999999,"5min":16.036999999999999,"15min":16.036999999999999},"last":0.40899999999999997}]},{"osd":4,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":8.0600000000000005,"5min":8.0600000000000005,"15min":8.0600000000000005},"last":0.62},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":8.2739999999999991,"5min":8.2739999999999991,"15min":8.2739999999999991},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":8.3320000000000007,"5min":8.3320000000000007,"15min":8.3320000000000007},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":8.0299999999999994,"5min":8.0299999999999994,"15min":8.0299999999999994},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.3030000000000008,"5min":8.3030000000000008,"15min":8.3030000000000008},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":8.3800000000000008,"5min":8.3800000000000008,"15min":8.3800000000000008},"last":0.74399999999999999}]},{"osd":7,"last update":"Fri Jun 23 18:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":8.3580000000000005,"5min":8.3580000000000005,"15min":8.3580000000000005},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":8.0839999999999996,"5min":8.0839999999999996,"15min":8.0839999999999996},"last":0.65100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":177564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.32400000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.40999999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":5.0899999999999999,"5min":5.0899999999999999,"15min":5.0899999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.55500000000000005}]},{"osd":4,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.503},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.47499999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46100000000000002}]},{"osd":7,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":178925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.504},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":4.359,"5min":4.359,"15min":4.359},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.65300000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.69899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":178925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.53400000000000003}]},{"osd":2,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.378},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.36399999999999999}]},{"osd":6,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.42199999999999999}]},{"osd":7,"last update":"Fri Jun 23 18:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.434}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":178925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58299999999999996}]},{"osd":2,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.37}]},{"osd":6,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47499999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.435},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:37:49.880 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-06-23T18:37:50.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.996+0000 7f797eed7700 1 Processor -- start 2023-06-23T18:37:50.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.996+0000 7f797eed7700 1 -- start start 2023-06-23T18:37:50.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.996+0000 7f797eed7700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f797815c840 0x7f79780b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:50.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.996+0000 7f797eed7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79780b0890 con 0x7f797806b6b0 2023-06-23T18:37:50.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.996+0000 7f797eed7700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79780d0090 con 0x7f79780cfc10 2023-06-23T18:37:50.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.996+0000 7f797eed7700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79780d0310 con 0x7f797815c840 2023-06-23T18:37:50.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f797d474700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f797806b6b0 0x7f79780ce530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:45594/0 (socket says 172.21.15.79:45594) 2023-06-23T18:37:50.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f797d474700 1 -- 172.21.15.79:0/4154573224 learned_addr learned my addr 172.21.15.79:0/4154573224 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:50.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f797cc73700 1 --2- 172.21.15.79:0/4154573224 >> v2:172.21.15.155:3300/0 conn(0x7f797815c840 0x7f79780b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:50.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200462544 0 0) 0x7f79780b0890 con 0x7f797806b6b0 2023-06-23T18:37:50.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79680035b0 con 0x7f797806b6b0 2023-06-23T18:37:50.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022819967 0 0) 0x7f79780d0090 con 0x7f79780cfc10 2023-06-23T18:37:50.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79780b0890 con 0x7f79780cfc10 2023-06-23T18:37:50.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1375282925 0 0) 0x7f79680035b0 con 0x7f797806b6b0 2023-06-23T18:37:50.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 >> v2:172.21.15.155:3300/0 conn(0x7f797815c840 msgr2=0x7f79780b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:50.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 --2- 172.21.15.79:0/4154573224 >> v2:172.21.15.155:3300/0 conn(0x7f797815c840 0x7f79780b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 >> v1:172.21.15.155:6789/0 conn(0x7f79780cfc10 legacy=0x7f797806afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:50.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79780b0ae0 con 0x7f797806b6b0 2023-06-23T18:37:50.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532881089 0 0) 0x7f7970001970 con 0x7f797806b6b0 2023-06-23T18:37:50.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 >> v1:172.21.15.79:6789/0 conn(0x7f797806b6b0 legacy=0x7f79780ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:50.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 --2- 172.21.15.79:0/4154573224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7968003be0 0x7f7968003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:50.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 --2- 172.21.15.79:0/4154573224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7968004850 0x7f7968004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:50.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 --2- 172.21.15.79:0/4154573224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7968005200 0x7f796800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:50.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79780d0310 con 0x7f7968005200 2023-06-23T18:37:50.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796800c030 con 0x7f7968003be0 2023-06-23T18:37:50.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796800c1f0 con 0x7f7968004850 2023-06-23T18:37:50.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f797d474700 1 --2- 172.21.15.79:0/4154573224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7968005200 0x7f796800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:50.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f797cc73700 1 --2- 172.21.15.79:0/4154573224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7968003be0 0x7f7968003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:50.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.997+0000 7f7977fff700 1 --2- 172.21.15.79:0/4154573224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7968004850 0x7f7968004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:50.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f797d474700 1 -- 172.21.15.79:0/4154573224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7968004850 msgr2=0x7f7968004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:50.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f797d474700 1 --2- 172.21.15.79:0/4154573224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7968004850 0x7f7968004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f797d474700 1 -- 172.21.15.79:0/4154573224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7968003be0 msgr2=0x7f7968003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:50.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f797d474700 1 --2- 172.21.15.79:0/4154573224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7968003be0 0x7f7968003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f797d474700 1 -- 172.21.15.79:0/4154573224 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79680035b0 con 0x7f7968005200 2023-06-23T18:37:50.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f797d474700 1 -- 172.21.15.79:0/4154573224 reap_dead start 2023-06-23T18:37:50.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f797d474700 1 --2- 172.21.15.79:0/4154573224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7968005200 0x7f796800ba10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f797000a680 tx=0x7f7968005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9541c11bcad58abd server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:50.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970007390 con 0x7f7968005200 2023-06-23T18:37:50.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7970007510 con 0x7f7968005200 2023-06-23T18:37:50.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f79777fe700 1 -- 172.21.15.79:0/4154573224 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79700078c0 con 0x7f7968005200 2023-06-23T18:37:50.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f797eed7700 1 -- 172.21.15.79:0/4154573224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7968005200 msgr2=0x7f796800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:50.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f797eed7700 1 --2- 172.21.15.79:0/4154573224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7968005200 0x7f796800ba10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f797000a680 tx=0x7f7968005880 comp rx=0 tx=0).stop 2023-06-23T18:37:50.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f797eed7700 1 -- 172.21.15.79:0/4154573224 shutdown_connections 2023-06-23T18:37:50.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f797eed7700 1 --2- 172.21.15.79:0/4154573224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7968005200 0x7f796800ba10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f797eed7700 1 -- 172.21.15.79:0/4154573224 >> 172.21.15.79:0/4154573224 conn(0x7f797806a7b0 msgr2=0x7f7978099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:50.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f797eed7700 1 -- 172.21.15.79:0/4154573224 shutdown_connections 2023-06-23T18:37:50.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:49.998+0000 7f797eed7700 1 -- 172.21.15.79:0/4154573224 wait complete. 2023-06-23T18:37:50.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.000+0000 7f797eed7700 1 Processor -- start 2023-06-23T18:37:50.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.000+0000 7f797eed7700 1 -- start start 2023-06-23T18:37:50.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.000+0000 7f797eed7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7968005200 0x7f797815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:50.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.000+0000 7f797eed7700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f797806f830 0x7f797806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:50.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797eed7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79780c7a30 0x7f79780c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:50.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797eed7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79780701e0 con 0x7f7968005200 2023-06-23T18:37:50.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797eed7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79780c83f0 con 0x7f79780c7a30 2023-06-23T18:37:50.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797eed7700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797815b340 con 0x7f797806f830 2023-06-23T18:37:50.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797cc73700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7968005200 0x7f797815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:50.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797cc73700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7968005200 0x7f797815a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42560/0 (socket says 172.21.15.79:42560) 2023-06-23T18:37:50.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797cc73700 1 -- 172.21.15.79:0/2607347304 learned_addr learned my addr 172.21.15.79:0/2607347304 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:50.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797d474700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79780c7a30 0x7f79780c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:50.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f7977fff700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f797806f830 0x7f797806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:50.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797cc73700 1 -- 172.21.15.79:0/2607347304 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f797806f830 msgr2=0x7f797806fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:50.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797cc73700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f797806f830 0x7f797806fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797cc73700 1 -- 172.21.15.79:0/2607347304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79780c7a30 msgr2=0x7f79780c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:50.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797cc73700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79780c7a30 0x7f79780c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797cc73700 1 -- 172.21.15.79:0/2607347304 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79780c8610 con 0x7f7968005200 2023-06-23T18:37:50.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797d474700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79780c7a30 0x7f79780c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:50.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f7977fff700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f797806f830 0x7f797806fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:37:50.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797cc73700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7968005200 0x7f797815a790 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f796c0119a0 tx=0x7f796c011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:50.022 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f79757fa700 1 -- 172.21.15.79:0/2607347304 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c005c30 con 0x7f7968005200 2023-06-23T18:37:50.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f79757fa700 1 -- 172.21.15.79:0/2607347304 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c005d90 con 0x7f7968005200 2023-06-23T18:37:50.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797eed7700 1 -- 172.21.15.79:0/2607347304 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79780c88f0 con 0x7f7968005200 2023-06-23T18:37:50.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f79757fa700 1 -- 172.21.15.79:0/2607347304 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c00f910 con 0x7f7968005200 2023-06-23T18:37:50.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.001+0000 7f797eed7700 1 -- 172.21.15.79:0/2607347304 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79780c8eb0 con 0x7f7968005200 2023-06-23T18:37:50.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.002+0000 7f797eed7700 1 -- 172.21.15.79:0/2607347304 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79780672c0 con 0x7f7968005200 2023-06-23T18:37:50.024 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.003+0000 7f79757fa700 1 -- 172.21.15.79:0/2607347304 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f796c00fa70 con 0x7f7968005200 2023-06-23T18:37:50.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.003+0000 7f79757fa700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f795c06ace0 0x7f795c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:50.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.003+0000 7f7977fff700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f795c06ace0 0x7f795c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:50.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.003+0000 7f79757fa700 1 -- 172.21.15.79:0/2607347304 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f796c0897d0 con 0x7f7968005200 2023-06-23T18:37:50.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.004+0000 7f7977fff700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f795c06ace0 0x7f795c06d190 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7964007fd0 tx=0x7f7964009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:50.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.008+0000 7f79757fa700 1 -- 172.21.15.79:0/2607347304 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f796c054b80 con 0x7f7968005200 2023-06-23T18:37:50.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.198+0000 7f797eed7700 1 -- 172.21.15.79:0/2607347304 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f797806bb60 con 0x7f7968005200 2023-06-23T18:37:50.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.202+0000 7f79757fa700 1 -- 172.21.15.79:0/2607347304 <== mon.0 v2:172.21.15.79:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f796c054b80 con 0x7f7968005200 2023-06-23T18:37:50.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.203+0000 7f79757fa700 1 -- 172.21.15.79:0/2607347304 <== mon.0 v2:172.21.15.79:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f796c0210c0 con 0x7f7968005200 2023-06-23T18:37:50.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.205+0000 7f797eed7700 1 -- 172.21.15.79:0/2607347304 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f795c06ace0 msgr2=0x7f795c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:50.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.205+0000 7f797eed7700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f795c06ace0 0x7f795c06d190 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7964007fd0 tx=0x7f7964009040 comp rx=0 tx=0).stop 2023-06-23T18:37:50.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.205+0000 7f797eed7700 1 -- 172.21.15.79:0/2607347304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7968005200 msgr2=0x7f797815a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:50.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.205+0000 7f797eed7700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7968005200 0x7f797815a790 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f796c0119a0 tx=0x7f796c011d60 comp rx=0 tx=0).stop 2023-06-23T18:37:50.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.205+0000 7f797eed7700 1 -- 172.21.15.79:0/2607347304 shutdown_connections 2023-06-23T18:37:50.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.205+0000 7f797eed7700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f795c06ace0 0x7f795c06d190 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.205+0000 7f797eed7700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79780c7a30 0x7f79780c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.205+0000 7f797eed7700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7968005200 0x7f797815a790 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.205+0000 7f797eed7700 1 --2- 172.21.15.79:0/2607347304 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f797806f830 0x7f797806fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.205+0000 7f797eed7700 1 -- 172.21.15.79:0/2607347304 >> 172.21.15.79:0/2607347304 conn(0x7f797806a7b0 msgr2=0x7f79780cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:50.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.205+0000 7f797eed7700 1 -- 172.21.15.79:0/2607347304 shutdown_connections 2023-06-23T18:37:50.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.205+0000 7f797eed7700 1 -- 172.21.15.79:0/2607347304 wait complete. 2023-06-23T18:37:50.222 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-06-23T18:37:50.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:37:50.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.337+0000 7f95473c6700 1 Processor -- start 2023-06-23T18:37:50.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.338+0000 7f95473c6700 1 -- start start 2023-06-23T18:37:50.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.338+0000 7f95473c6700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9540159170 0x7f95400cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:50.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.338+0000 7f95473c6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95400cd900 0x7f95400ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:50.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.338+0000 7f95473c6700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954015cd50 con 0x7f95400ce620 2023-06-23T18:37:50.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.338+0000 7f95473c6700 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95400cd1e0 con 0x7f9540159170 2023-06-23T18:37:50.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.338+0000 7f95473c6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95400cdda0 con 0x7f95400cd900 2023-06-23T18:37:50.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.339+0000 7f9545162700 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9540159170 0x7f95400cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:50.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.339+0000 7f9544961700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95400cd900 0x7f95400ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:50.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.339+0000 7f9545963700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f95400ce620 0x7f9540071490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.79:53882/0 (socket says 172.21.15.79:53882) 2023-06-23T18:37:50.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.339+0000 7f9545963700 1 -- 172.21.15.79:0/1261562783 learned_addr learned my addr 172.21.15.79:0/1261562783 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-23T18:37:50.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.339+0000 7f9544961700 1 -- 172.21.15.79:0/1261562783 >> v2:172.21.15.155:3300/0 conn(0x7f9540159170 msgr2=0x7f95400cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:50.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.339+0000 7f9544961700 1 --2- 172.21.15.79:0/1261562783 >> v2:172.21.15.155:3300/0 conn(0x7f9540159170 0x7f95400cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.339+0000 7f9544961700 1 -- 172.21.15.79:0/1261562783 >> v1:172.21.15.155:6789/0 conn(0x7f95400ce620 legacy=0x7f9540071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:50.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.339+0000 7f9544961700 1 -- 172.21.15.79:0/1261562783 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954015cfa0 con 0x7f95400cd900 2023-06-23T18:37:50.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.339+0000 7f9545162700 1 --2- 172.21.15.79:0/1261562783 >> v2:172.21.15.155:3300/0 conn(0x7f9540159170 0x7f95400cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:50.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.340+0000 7f9544961700 1 --2- 172.21.15.79:0/1261562783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95400cd900 0x7f95400ce0e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f953c009840 tx=0x7f953c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b625e7560c61e44 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:50.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.340+0000 7f9537fff700 1 -- 172.21.15.79:0/1261562783 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c011830 con 0x7f95400cd900 2023-06-23T18:37:50.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.340+0000 7f9537fff700 1 -- 172.21.15.79:0/1261562783 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f953c011990 con 0x7f95400cd900 2023-06-23T18:37:50.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.340+0000 7f9537fff700 1 -- 172.21.15.79:0/1261562783 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c011d40 con 0x7f95400cd900 2023-06-23T18:37:50.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.340+0000 7f95473c6700 1 -- 172.21.15.79:0/1261562783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95400cd900 msgr2=0x7f95400ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:50.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.340+0000 7f95473c6700 1 --2- 172.21.15.79:0/1261562783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95400cd900 0x7f95400ce0e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f953c009840 tx=0x7f953c009b50 comp rx=0 tx=0).stop 2023-06-23T18:37:50.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.340+0000 7f95473c6700 1 -- 172.21.15.79:0/1261562783 shutdown_connections 2023-06-23T18:37:50.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.340+0000 7f95473c6700 1 --2- 172.21.15.79:0/1261562783 >> v2:172.21.15.155:3300/0 conn(0x7f9540159170 0x7f95400cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.340+0000 7f95473c6700 1 --2- 172.21.15.79:0/1261562783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95400cd900 0x7f95400ce0e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.340+0000 7f95473c6700 1 -- 172.21.15.79:0/1261562783 >> 172.21.15.79:0/1261562783 conn(0x7f954006d660 msgr2=0x7f954006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:50.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.340+0000 7f95473c6700 1 -- 172.21.15.79:0/1261562783 shutdown_connections 2023-06-23T18:37:50.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.340+0000 7f95473c6700 1 -- 172.21.15.79:0/1261562783 wait complete. 2023-06-23T18:37:50.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.342+0000 7f95473c6700 1 Processor -- start 2023-06-23T18:37:50.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.342+0000 7f95473c6700 1 -- start start 2023-06-23T18:37:50.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.342+0000 7f95473c6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95400cd900 0x7f9540144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:50.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.342+0000 7f95473c6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95400ce620 0x7f95400c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:50.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.342+0000 7f95473c6700 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9540159170 0x7f954015b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:50.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.342+0000 7f95473c6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9540145120 con 0x7f95400ce620 2023-06-23T18:37:50.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.342+0000 7f95473c6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95400c7380 con 0x7f95400cd900 2023-06-23T18:37:50.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.342+0000 7f95473c6700 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95400c7660 con 0x7f9540159170 2023-06-23T18:37:50.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f9544961700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95400ce620 0x7f95400c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:50.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f9544961700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95400ce620 0x7f95400c68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42578/0 (socket says 172.21.15.79:42578) 2023-06-23T18:37:50.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f9544961700 1 -- 172.21.15.79:0/3898928787 learned_addr learned my addr 172.21.15.79:0/3898928787 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-23T18:37:50.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f9545162700 1 --2- 172.21.15.79:0/3898928787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95400cd900 0x7f9540144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:50.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f9545963700 1 --2- 172.21.15.79:0/3898928787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9540159170 0x7f954015b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:50.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f9544961700 1 -- 172.21.15.79:0/3898928787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9540159170 msgr2=0x7f954015b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:50.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f9544961700 1 --2- 172.21.15.79:0/3898928787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9540159170 0x7f954015b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f9544961700 1 -- 172.21.15.79:0/3898928787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95400cd900 msgr2=0x7f9540144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:50.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f9544961700 1 --2- 172.21.15.79:0/3898928787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95400cd900 0x7f9540144be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f9544961700 1 -- 172.21.15.79:0/3898928787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95300094a0 con 0x7f95400ce620 2023-06-23T18:37:50.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f9544961700 1 --2- 172.21.15.79:0/3898928787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95400ce620 0x7f95400c68c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f953c009fd0 tx=0x7f953c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:50.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f9535ffb700 1 -- 172.21.15.79:0/3898928787 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c0036a0 con 0x7f95400ce620 2023-06-23T18:37:50.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f9535ffb700 1 -- 172.21.15.79:0/3898928787 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f953c01f070 con 0x7f95400ce620 2023-06-23T18:37:50.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f95473c6700 1 -- 172.21.15.79:0/3898928787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953c0094a0 con 0x7f95400ce620 2023-06-23T18:37:50.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f9535ffb700 1 -- 172.21.15.79:0/3898928787 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c003a80 con 0x7f95400ce620 2023-06-23T18:37:50.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.343+0000 7f95473c6700 1 -- 172.21.15.79:0/3898928787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95400c7c20 con 0x7f95400ce620 2023-06-23T18:37:50.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.344+0000 7f95473c6700 1 -- 172.21.15.79:0/3898928787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95400c9e20 con 0x7f95400ce620 2023-06-23T18:37:50.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.345+0000 7f9535ffb700 1 -- 172.21.15.79:0/3898928787 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f953c003be0 con 0x7f95400ce620 2023-06-23T18:37:50.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.345+0000 7f9535ffb700 1 --2- 172.21.15.79:0/3898928787 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f952406ace0 0x7f952406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:50.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.345+0000 7f9535ffb700 1 -- 172.21.15.79:0/3898928787 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f953c021070 con 0x7f95400ce620 2023-06-23T18:37:50.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.345+0000 7f9545162700 1 --2- 172.21.15.79:0/3898928787 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f952406ace0 0x7f952406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:50.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.345+0000 7f9545162700 1 --2- 172.21.15.79:0/3898928787 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f952406ace0 0x7f952406d190 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9530009cd0 tx=0x7f9530009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:50.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.349+0000 7f9535ffb700 1 -- 172.21.15.79:0/3898928787 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f953c055d80 con 0x7f95400ce620 2023-06-23T18:37:50.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.581+0000 7f95473c6700 1 -- 172.21.15.79:0/3898928787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95400b5770 con 0x7f95400ce620 2023-06-23T18:37:50.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.581+0000 7f9535ffb700 1 -- 172.21.15.79:0/3898928787 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f953c015090 con 0x7f95400ce620 2023-06-23T18:37:50.587 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-23T18:37:50.587 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3101032943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":3101032943},{"type":"v1","addr":"172.21.15.79:6837","nonce":3101032943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2056827644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2056827644},{"type":"v1","addr":"172.21.15.155:6833","nonce":2056827644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1835571378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1835571378},{"type":"v1","addr":"172.21.15.155:6835","nonce":1835571378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/2532453606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2532453606},{"type":"v1","addr":"172.21.15.79:6836","nonce":2532453606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-06-23T18:37:50.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.583+0000 7f95473c6700 1 -- 172.21.15.79:0/3898928787 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f952406ace0 msgr2=0x7f952406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:50.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.583+0000 7f95473c6700 1 --2- 172.21.15.79:0/3898928787 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f952406ace0 0x7f952406d190 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9530009cd0 tx=0x7f9530009210 comp rx=0 tx=0).stop 2023-06-23T18:37:50.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.583+0000 7f95473c6700 1 -- 172.21.15.79:0/3898928787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95400ce620 msgr2=0x7f95400c68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:50.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.583+0000 7f95473c6700 1 --2- 172.21.15.79:0/3898928787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95400ce620 0x7f95400c68c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f953c009fd0 tx=0x7f953c004750 comp rx=0 tx=0).stop 2023-06-23T18:37:50.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.583+0000 7f95473c6700 1 -- 172.21.15.79:0/3898928787 shutdown_connections 2023-06-23T18:37:50.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.583+0000 7f95473c6700 1 --2- 172.21.15.79:0/3898928787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9540159170 0x7f954015b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.583+0000 7f95473c6700 1 --2- 172.21.15.79:0/3898928787 >> [v2:172.21.15.79:6832/1804789972,v1:172.21.15.79:6833/1804789972] conn(0x7f952406ace0 0x7f952406d190 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.583+0000 7f95473c6700 1 --2- 172.21.15.79:0/3898928787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95400cd900 0x7f9540144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.583+0000 7f95473c6700 1 --2- 172.21.15.79:0/3898928787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95400ce620 0x7f95400c68c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:50.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.583+0000 7f95473c6700 1 -- 172.21.15.79:0/3898928787 >> 172.21.15.79:0/3898928787 conn(0x7f954006d660 msgr2=0x7f9540069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:50.590 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.583+0000 7f95473c6700 1 -- 172.21.15.79:0/3898928787 shutdown_connections 2023-06-23T18:37:50.591 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-23T18:37:50.583+0000 7f95473c6700 1 -- 172.21.15.79:0/3898928787 wait complete. 2023-06-23T18:37:50.591 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 18 2023-06-23T18:37:50.600 DEBUG:tasks.cephfs.filesystem:already dead... 2023-06-23T18:37:50.601 INFO:teuthology.misc:Shutting down mds daemons... 2023-06-23T18:37:50.601 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-06-23T18:37:50.601 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:50.610 INFO:tasks.ceph.mds.a:Stopped 2023-06-23T18:37:50.610 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-06-23T18:37:50.610 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:50.658 INFO:tasks.ceph.mds.c:Stopped 2023-06-23T18:37:50.659 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-23T18:37:50.659 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:50.670 INFO:tasks.ceph.mds.b:Stopped 2023-06-23T18:37:50.671 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-23T18:37:50.671 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:50.722 INFO:tasks.ceph.mds.d:Stopped 2023-06-23T18:37:50.722 INFO:teuthology.misc:Shutting down osd daemons... 2023-06-23T18:37:50.723 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-06-23T18:37:50.723 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:50.754 INFO:tasks.ceph.osd.0:Stopped 2023-06-23T18:37:50.755 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-06-23T18:37:50.755 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:50.822 INFO:tasks.ceph.osd.1:Stopped 2023-06-23T18:37:50.823 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-06-23T18:37:50.823 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:50.892 INFO:tasks.ceph.osd.2:Stopped 2023-06-23T18:37:50.893 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-06-23T18:37:50.893 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:50.964 INFO:tasks.ceph.osd.3:Stopped 2023-06-23T18:37:50.965 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-06-23T18:37:50.965 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:51.007 INFO:tasks.ceph.osd.4:Stopped 2023-06-23T18:37:51.008 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-06-23T18:37:51.009 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:51.050 INFO:tasks.ceph.osd.5:Stopped 2023-06-23T18:37:51.051 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-06-23T18:37:51.051 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:51.096 INFO:tasks.ceph.osd.6:Stopped 2023-06-23T18:37:51.096 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-06-23T18:37:51.097 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:51.130 INFO:tasks.ceph.osd.7:Stopped 2023-06-23T18:37:51.131 INFO:teuthology.misc:Shutting down mgr daemons... 2023-06-23T18:37:51.132 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-06-23T18:37:51.133 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:51.156 INFO:tasks.ceph.mgr.y:Stopped 2023-06-23T18:37:51.157 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-06-23T18:37:51.158 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:51.195 INFO:tasks.ceph.mgr.x:Stopped 2023-06-23T18:37:51.196 INFO:teuthology.misc:Shutting down mon daemons... 2023-06-23T18:37:51.197 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-06-23T18:37:51.197 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:51.207 INFO:tasks.ceph.mon.a:Stopped 2023-06-23T18:37:51.208 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-06-23T18:37:51.208 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:51.247 INFO:tasks.ceph.mon.b:Stopped 2023-06-23T18:37:51.248 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-06-23T18:37:51.248 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:37:51.299 INFO:tasks.ceph.mon.c:Stopped 2023-06-23T18:37:51.299 INFO:tasks.ceph:Checking cluster log for badness... 2023-06-23T18:37:51.300 DEBUG:teuthology.orchestra.run.smithi079:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-06-23T18:37:51.331 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:37:51.332 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-06-23T18:37:51.726 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:37:51.727 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-06-23T18:37:51.826 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:37:51.826 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-06-23T18:37:51.922 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:37:51.922 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-06-23T18:37:51.991 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:37:51.992 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-06-23T18:37:52.194 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:37:52.194 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-06-23T18:37:52.300 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:37:52.301 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-06-23T18:37:52.398 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:37:52.399 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-06-23T18:37:52.485 INFO:tasks.ceph:Archiving mon data... 2023-06-23T18:37:52.487 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886/data/mon.a.tgz 2023-06-23T18:37:52.488 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2023-06-23T18:37:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.vv1rsZSVDX 2023-06-23T18:37:52.507 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.vv1rsZSVDX 2023-06-23T18:37:52.634 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.vv1rsZSVDX 2023-06-23T18:37:52.719 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.vv1rsZSVDX is 282KB 2023-06-23T18:37:52.789 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.vv1rsZSVDX 2023-06-23T18:37:52.806 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886/data/mon.b.tgz 2023-06-23T18:37:52.807 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2023-06-23T18:37:52.827 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.eLTYtqsj3R 2023-06-23T18:37:52.828 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.eLTYtqsj3R 2023-06-23T18:37:52.973 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.eLTYtqsj3R 2023-06-23T18:37:53.061 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.eLTYtqsj3R is 379KB 2023-06-23T18:37:53.131 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.eLTYtqsj3R 2023-06-23T18:37:53.147 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886/data/mon.c.tgz 2023-06-23T18:37:53.148 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2023-06-23T18:37:53.203 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.aen7WMupsD 2023-06-23T18:37:53.204 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.aen7WMupsD 2023-06-23T18:37:53.336 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.aen7WMupsD 2023-06-23T18:37:53.422 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.aen7WMupsD is 379KB 2023-06-23T18:37:53.490 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.aen7WMupsD 2023-06-23T18:37:53.506 INFO:tasks.ceph:Cleaning ceph cluster... 2023-06-23T18:37:53.507 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-06-23T18:37:53.510 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-06-23T18:37:53.572 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-06-23T18:37:53.573 DEBUG:teuthology.orchestra.run.smithi079:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-06-23T18:37:53.579 DEBUG:teuthology.orchestra.run.smithi155:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-06-23T18:37:53.624 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-06-23T18:37:53.648 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-06-23T18:37:53.655 INFO:tasks.ceph:Archiving crash dumps... 2023-06-23T18:37:53.657 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886/remote/smithi079/crash 2023-06-23T18:37:53.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-06-23T18:37:53.687 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886/remote/smithi155/crash 2023-06-23T18:37:53.689 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-06-23T18:37:53.720 INFO:tasks.ceph:Shutting down logrotate 2023-06-23T18:37:53.721 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:37:53.733 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:37:53.839 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-06-23T18:37:53.868 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-06-23T18:37:53.894 INFO:tasks.ceph:Compressing logs... 2023-06-23T18:37:53.895 DEBUG:teuthology.orchestra.run.smithi079:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-06-23T18:37:53.910 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-06-23T18:37:53.936 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111182.log 2023-06-23T18:37:53.937 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.110131.log 2023-06-23T18:37:53.937 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 /var/log/ceph/ceph-client.admin.111182.log: --verbose -- /var/log/ceph/ceph-client.admin.118571.log 2023-06-23T18:37:53.972 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-06-23T18:37:53.973 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.107975.log 2023-06-23T18:37:53.973 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-06-23T18:37:53.974 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.3.107975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100540.log 2023-06-23T18:37:53.974 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.107382.log 2023-06-23T18:37:53.974 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.100540.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-06-23T18:37:53.974 INFO:teuthology.orchestra.run.smithi155.stderr: 87.1% 91.2% -- replaced with /var/log/ceph/ceph-client.admin.100540.log.gz -- replaced with /var/log/ceph/ceph-client.3.107975.log.gz 2023-06-23T18:37:53.975 INFO:teuthology.orchestra.run.smithi155.stderr: 2023-06-23T18:37:53.975 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.107382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.108822.log 2023-06-23T18:37:53.975 INFO:teuthology.orchestra.run.smithi155.stderr: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2023-06-23T18:37:53.976 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100631.log 2023-06-23T18:37:53.976 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.1.108822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100774.log 2023-06-23T18:37:53.976 INFO:teuthology.orchestra.run.smithi155.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.1.107382.log.gz 2023-06-23T18:37:53.976 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.100631.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-06-23T18:37:53.977 INFO:teuthology.orchestra.run.smithi155.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100631.log.gz 2023-06-23T18:37:53.977 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.100774.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-06-23T18:37:53.977 INFO:teuthology.orchestra.run.smithi155.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.1.108822.log.gz 2023-06-23T18:37:53.977 INFO:teuthology.orchestra.run.smithi155.stderr: 88.4%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.100774.log.gz--verbose 2023-06-23T18:37:53.978 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mds.d.log: -- /var/log/ceph/ceph-osd.4.log 2023-06-23T18:37:53.978 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100586.log 2023-06-23T18:37:53.978 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-06-23T18:37:53.978 INFO:teuthology.orchestra.run.smithi155.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-06-23T18:37:53.979 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.100586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100677.log 2023-06-23T18:37:53.979 INFO:teuthology.orchestra.run.smithi155.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100586.log.gz 2023-06-23T18:37:53.980 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-06-23T18:37:53.981 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.100677.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.100677.log.gz 2023-06-23T18:37:53.982 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mon.c.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-06-23T18:37:53.982 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-06-23T18:37:53.982 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100730.log 2023-06-23T18:37:53.987 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-client.admin.100730.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100730.log.gz 2023-06-23T18:37:53.988 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100494.log 2023-06-23T18:37:53.988 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.100494.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100494.log.gz 2023-06-23T18:37:53.988 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-06-23T18:37:53.988 INFO:teuthology.orchestra.run.smithi155.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-06-23T18:37:53.989 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-06-23T18:37:53.989 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-mds.b.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-06-23T18:37:53.989 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100447.log 2023-06-23T18:37:53.989 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.100447.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100447.log.gz 2023-06-23T18:37:53.998 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.0.110131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112573.log 2023-06-23T18:37:53.998 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111182.log.gz 2023-06-23T18:37:53.999 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114876.log 2023-06-23T18:37:53.999 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116170.log 2023-06-23T18:37:53.999 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118571.log.gz 2023-06-23T18:37:53.999 INFO:teuthology.orchestra.run.smithi079.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.0.110131.log.gz 2023-06-23T18:37:53.999 INFO:teuthology.orchestra.run.smithi079.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112573.log.gz 2023-06-23T18:37:54.000 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114462.log 2023-06-23T18:37:54.000 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116170.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114876.log.gz 2023-06-23T18:37:54.000 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117459.log 2023-06-23T18:37:54.000 INFO:teuthology.orchestra.run.smithi079.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116170.log.gz 2023-06-23T18:37:54.000 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119211.log 2023-06-23T18:37:54.001 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117459.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114462.log.gz 2023-06-23T18:37:54.001 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111275.log 2023-06-23T18:37:54.001 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119211.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117459.log.gz 2023-06-23T18:37:54.001 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119923.log 2023-06-23T18:37:54.001 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111275.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119211.log.gz 2023-06-23T18:37:54.002 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113684.log 2023-06-23T18:37:54.002 INFO:teuthology.orchestra.run.smithi079.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111275.log.gz 2023-06-23T18:37:54.002 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111229.log 2023-06-23T18:37:54.002 INFO:teuthology.orchestra.run.smithi079.stderr: 88.2%/var/log/ceph/ceph-client.admin.113684.log: -- replaced with /var/log/ceph/ceph-client.admin.119923.log.gz 2023-06-23T18:37:54.002 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106416.log 2023-06-23T18:37:54.003 INFO:teuthology.orchestra.run.smithi079.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113684.log.gz 2023-06-23T18:37:54.003 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111229.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-06-23T18:37:54.003 INFO:teuthology.orchestra.run.smithi079.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111229.log.gz 2023-06-23T18:37:54.003 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.106416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113259.log 2023-06-23T18:37:54.003 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.2.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106416.log.gz 2023-06-23T18:37:54.004 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114784.log 2023-06-23T18:37:54.004 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107713.log 2023-06-23T18:37:54.004 INFO:teuthology.orchestra.run.smithi079.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113259.log.gz 2023-06-23T18:37:54.004 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111136.log 2023-06-23T18:37:54.004 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112711.log 2023-06-23T18:37:54.005 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114784.log.gz 2023-06-23T18:37:54.005 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111136.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107713.log.gz 2023-06-23T18:37:54.005 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108849.log 2023-06-23T18:37:54.005 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111136.log.gz 2023-06-23T18:37:54.005 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108757.log 2023-06-23T18:37:54.006 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111827.log 2023-06-23T18:37:54.006 INFO:teuthology.orchestra.run.smithi079.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112711.log.gz 2023-06-23T18:37:54.006 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108757.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.108849.log.gz 2023-06-23T18:37:54.006 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118935.log 2023-06-23T18:37:54.006 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108757.log.gz 2023-06-23T18:37:54.007 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112799.log 2023-06-23T18:37:54.007 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118935.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111827.log.gz 2023-06-23T18:37:54.007 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119165.log 2023-06-23T18:37:54.007 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112799.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118935.log.gz 2023-06-23T18:37:54.007 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119784.log 2023-06-23T18:37:54.008 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119165.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112799.log.gz 2023-06-23T18:37:54.008 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114324.log 2023-06-23T18:37:54.008 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119784.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119165.log.gz 2023-06-23T18:37:54.008 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114094.log 2023-06-23T18:37:54.008 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114324.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119784.log.gz 2023-06-23T18:37:54.009 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118525.log 2023-06-23T18:37:54.009 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114324.log.gz 2023-06-23T18:37:54.009 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114094.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-06-23T18:37:54.009 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118525.log: 89.1%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.114094.log.gz--verbose 2023-06-23T18:37:54.009 INFO:teuthology.orchestra.run.smithi079.stderr:-- /var/log/ceph/ceph-client.admin.115014.log 2023-06-23T18:37:54.009 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.118525.log.gz 2023-06-23T18:37:54.010 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109171.log 2023-06-23T18:37:54.010 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118092.log 2023-06-23T18:37:54.010 INFO:teuthology.orchestra.run.smithi079.stderr: 84.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-06-23T18:37:54.010 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109171.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115014.log.gz 2023-06-23T18:37:54.010 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107356.log 2023-06-23T18:37:54.011 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118092.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109171.log.gz 2023-06-23T18:37:54.011 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108803.log 2023-06-23T18:37:54.011 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118092.log.gz 2023-06-23T18:37:54.011 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106782.log 2023-06-23T18:37:54.011 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108803.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107356.log.gz 2023-06-23T18:37:54.012 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110918.log 2023-06-23T18:37:54.012 INFO:teuthology.orchestra.run.smithi079.stderr: 88.7%/var/log/ceph/ceph-client.admin.106782.log: -- replaced with /var/log/ceph/ceph-client.admin.108803.log.gz 2023-06-23T18:37:54.012 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106540.log 2023-06-23T18:37:54.012 INFO:teuthology.orchestra.run.smithi079.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.106782.log.gz/var/log/ceph/ceph-client.admin.110918.log: 2023-06-23T18:37:54.012 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108941.log 2023-06-23T18:37:54.013 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.106540.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110918.log.gz 2023-06-23T18:37:54.013 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119303.log 2023-06-23T18:37:54.013 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108941.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.106540.log.gz 2023-06-23T18:37:54.013 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115955.log 2023-06-23T18:37:54.013 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108941.log.gz 2023-06-23T18:37:54.014 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107717.log 2023-06-23T18:37:54.014 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107310.log 2023-06-23T18:37:54.014 INFO:teuthology.orchestra.run.smithi079.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119303.log.gz 2023-06-23T18:37:54.014 INFO:teuthology.orchestra.run.smithi079.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115955.log.gz 2023-06-23T18:37:54.014 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106589.log 2023-06-23T18:37:54.015 INFO:teuthology.orchestra.run.smithi079.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107717.log.gz 2023-06-23T18:37:54.015 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112619.log 2023-06-23T18:37:54.015 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.106589.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107310.log.gz 2023-06-23T18:37:54.015 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107448.log 2023-06-23T18:37:54.015 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112619.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.106589.log.gz 2023-06-23T18:37:54.016 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108453.log 2023-06-23T18:37:54.016 INFO:teuthology.orchestra.run.smithi079.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.112619.log.gz 2023-06-23T18:37:54.016 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117367.log 2023-06-23T18:37:54.016 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108453.log: 89.3%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.107448.log.gz--verbose 2023-06-23T18:37:54.016 INFO:teuthology.orchestra.run.smithi079.stderr:-- /var/log/ceph/ceph-osd.0.log 2023-06-23T18:37:54.017 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108453.log.gz 2023-06-23T18:37:54.017 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119119.log 2023-06-23T18:37:54.017 INFO:teuthology.orchestra.run.smithi079.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117367.log.gz 2023-06-23T18:37:54.017 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108086.log 2023-06-23T18:37:54.017 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107218.log 2023-06-23T18:37:54.018 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119119.log.gz 2023-06-23T18:37:54.018 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113167.log 2023-06-23T18:37:54.018 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107218.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108086.log.gz 2023-06-23T18:37:54.018 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115907.log 2023-06-23T18:37:54.018 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113167.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107218.log.gz 2023-06-23T18:37:54.019 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107715.log 2023-06-23T18:37:54.019 INFO:teuthology.orchestra.run.smithi079.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113167.log.gz 2023-06-23T18:37:54.019 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113776.log 2023-06-23T18:37:54.019 INFO:teuthology.orchestra.run.smithi079.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115907.log.gz 2023-06-23T18:37:54.019 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118889.log 2023-06-23T18:37:54.019 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107715.log: /var/log/ceph/ceph-client.admin.113776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119579.log 2023-06-23T18:37:54.020 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107715.log.gz 2023-06-23T18:37:54.020 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113776.log.gz 2023-06-23T18:37:54.020 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110964.log 2023-06-23T18:37:54.020 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115060.log 2023-06-23T18:37:54.021 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119579.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118889.log.gz 2023-06-23T18:37:54.021 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.110964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114416.log 2023-06-23T18:37:54.021 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119579.log.gz 2023-06-23T18:37:54.021 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115060.log: gzip 88.2%-5 -- replaced with /var/log/ceph/ceph-client.admin.110964.log.gz--verbose 2023-06-23T18:37:54.021 INFO:teuthology.orchestra.run.smithi079.stderr:-- /var/log/ceph/ceph-mds.a.log 2023-06-23T18:37:54.021 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114416.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115060.log.gz 2023-06-23T18:37:54.022 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117777.log 2023-06-23T18:37:54.022 INFO:teuthology.orchestra.run.smithi079.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114416.log.gz 2023-06-23T18:37:54.022 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119877.log 2023-06-23T18:37:54.022 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.117777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108458.log 2023-06-23T18:37:54.022 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119877.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117777.log.gz 2023-06-23T18:37:54.023 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108458.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119877.log.gz 2023-06-23T18:37:54.023 INFO:teuthology.orchestra.run.smithi079.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108458.log.gz 2023-06-23T18:37:54.023 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108087.log 2023-06-23T18:37:54.023 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109447.log 2023-06-23T18:37:54.023 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119625.log 2023-06-23T18:37:54.024 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113435.log 2023-06-23T18:37:54.024 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108087.log.gz 2023-06-23T18:37:54.024 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114048.log 2023-06-23T18:37:54.024 INFO:teuthology.orchestra.run.smithi079.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.119625.log.gz 2023-06-23T18:37:54.024 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113435.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.109447.log.gz 2023-06-23T18:37:54.025 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113121.log 2023-06-23T18:37:54.025 INFO:teuthology.orchestra.run.smithi079.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113435.log.gz 2023-06-23T18:37:54.025 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114048.log: /var/log/ceph/ceph-client.admin.113121.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114048.log.gz 2023-06-23T18:37:54.025 INFO:teuthology.orchestra.run.smithi079.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113121.log.gz 2023-06-23T18:37:54.025 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111367.log 2023-06-23T18:37:54.026 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108088.log 2023-06-23T18:37:54.026 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116057.log 2023-06-23T18:37:54.026 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114554.log 2023-06-23T18:37:54.026 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111367.log.gz 2023-06-23T18:37:54.026 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1%/var/log/ceph/ceph-client.admin.116057.log: -- replaced with /var/log/ceph/ceph-client.admin.108088.log.gz 2023-06-23T18:37:54.027 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107124.log 2023-06-23T18:37:54.027 INFO:teuthology.orchestra.run.smithi079.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116057.log.gz 2023-06-23T18:37:54.027 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114554.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-06-23T18:37:54.027 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118138.log 2023-06-23T18:37:54.027 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107124.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114554.log.gz 2023-06-23T18:37:54.028 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.3.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107124.log.gz 2023-06-23T18:37:54.028 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118138.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118138.log.gz 2023-06-23T18:37:54.028 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118230.log 2023-06-23T18:37:54.028 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107718.log 2023-06-23T18:37:54.028 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118230.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118230.log.gz 2023-06-23T18:37:54.029 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107718.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107718.log.gz 2023-06-23T18:37:54.029 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116264.log 2023-06-23T18:37:54.029 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115171.log 2023-06-23T18:37:54.029 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116264.log: /var/log/ceph/ceph-client.admin.115171.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116264.log.gz 2023-06-23T18:37:54.029 INFO:teuthology.orchestra.run.smithi079.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.115171.log.gz 2023-06-23T18:37:54.030 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111643.log 2023-06-23T18:37:54.030 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112527.log 2023-06-23T18:37:54.030 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111643.log: /var/log/ceph/ceph-client.admin.112527.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111643.log.gz 2023-06-23T18:37:54.030 INFO:teuthology.orchestra.run.smithi079.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.112527.log.gz 2023-06-23T18:37:54.030 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116494.log 2023-06-23T18:37:54.031 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117593.log 2023-06-23T18:37:54.031 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117321.log 2023-06-23T18:37:54.031 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5%/var/log/ceph/ceph-client.admin.117593.log: -- replaced with /var/log/ceph/ceph-client.admin.116494.log.gz 2023-06-23T18:37:54.031 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117321.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117593.log.gz 2023-06-23T18:37:54.031 INFO:teuthology.orchestra.run.smithi079.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117321.log.gz 2023-06-23T18:37:54.032 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111551.log 2023-06-23T18:37:54.032 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115401.log 2023-06-23T18:37:54.032 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113075.log 2023-06-23T18:37:54.032 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111551.log.gz 2023-06-23T18:37:54.032 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114968.log 2023-06-23T18:37:54.032 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115401.log.gz 2023-06-23T18:37:54.033 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112983.log 2023-06-23T18:37:54.033 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114968.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.113075.log.gz 2023-06-23T18:37:54.033 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114968.log.gz 2023-06-23T18:37:54.033 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109217.log 2023-06-23T18:37:54.033 INFO:teuthology.orchestra.run.smithi079.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.112983.log.gz 2023-06-23T18:37:54.034 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111090.log 2023-06-23T18:37:54.034 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109217.log: /var/log/ceph/ceph-client.admin.111090.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109217.log.gz 2023-06-23T18:37:54.034 INFO:teuthology.orchestra.run.smithi079.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111090.log.gz 2023-06-23T18:37:54.034 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-06-23T18:37:54.034 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112665.log 2023-06-23T18:37:54.035 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.112665.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112665.log.gz 2023-06-23T18:37:54.035 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118981.log 2023-06-23T18:37:54.035 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106930.log 2023-06-23T18:37:54.035 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118981.log: 88.5%/var/log/ceph/ceph-client.admin.106930.log: -- replaced with /var/log/ceph/ceph-client.admin.118981.log.gz 2023-06-23T18:37:54.035 INFO:teuthology.orchestra.run.smithi079.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106930.log.gz 2023-06-23T18:37:54.036 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109263.log 2023-06-23T18:37:54.036 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114002.log 2023-06-23T18:37:54.036 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109263.log: /var/log/ceph/ceph-client.admin.114002.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109263.log.gz 2023-06-23T18:37:54.036 INFO:teuthology.orchestra.run.smithi079.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114002.log.gz 2023-06-23T18:37:54.036 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107402.log 2023-06-23T18:37:54.037 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119349.log 2023-06-23T18:37:54.037 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107402.log: /var/log/ceph/ceph-client.admin.119349.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107402.log.gz 2023-06-23T18:37:54.037 INFO:teuthology.orchestra.run.smithi079.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119349.log.gz 2023-06-23T18:37:54.037 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106874.log 2023-06-23T18:37:54.037 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117413.log 2023-06-23T18:37:54.038 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.106874.log: /var/log/ceph/ceph-client.admin.117413.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.106874.log.gz 2023-06-23T18:37:54.038 INFO:teuthology.orchestra.run.smithi079.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117413.log.gz 2023-06-23T18:37:54.038 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115124.log 2023-06-23T18:37:54.038 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114692.log 2023-06-23T18:37:54.038 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115124.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.115124.log.gz 2023-06-23T18:37:54.039 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114692.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114692.log.gz 2023-06-23T18:37:54.039 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117275.log 2023-06-23T18:37:54.039 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108297.log 2023-06-23T18:37:54.039 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117275.log: /var/log/ceph/ceph-client.admin.108297.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117275.log.gz 2023-06-23T18:37:54.039 INFO:teuthology.orchestra.run.smithi079.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108297.log.gz 2023-06-23T18:37:54.040 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111919.log 2023-06-23T18:37:54.040 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118617.log 2023-06-23T18:37:54.040 INFO:teuthology.orchestra.run.smithi079.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-06-23T18:37:54.040 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112460.log 2023-06-23T18:37:54.040 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118617.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111919.log.gz 2023-06-23T18:37:54.041 INFO:teuthology.orchestra.run.smithi079.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118617.log.gz 2023-06-23T18:37:54.041 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.110779.log 2023-06-23T18:37:54.041 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115307.log 2023-06-23T18:37:54.041 INFO:teuthology.orchestra.run.smithi079.stderr: 87.9%/var/log/ceph/ceph-client.2.110779.log: -- replaced with /var/log/ceph/ceph-client.admin.112460.log.gz 2023-06-23T18:37:54.041 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115307.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115307.log.gz 2023-06-23T18:37:54.042 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118663.log 2023-06-23T18:37:54.042 INFO:teuthology.orchestra.run.smithi079.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.2.110779.log.gz 2023-06-23T18:37:54.042 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114232.log 2023-06-23T18:37:54.042 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118663.log: /var/log/ceph/ceph-client.admin.114232.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118663.log.gz 2023-06-23T18:37:54.042 INFO:teuthology.orchestra.run.smithi079.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114232.log.gz 2023-06-23T18:37:54.043 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109309.log 2023-06-23T18:37:54.043 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111459.log 2023-06-23T18:37:54.043 INFO:teuthology.orchestra.run.smithi079.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-06-23T18:37:54.043 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109309.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.109309.log.gz 2023-06-23T18:37:54.043 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111459.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111459.log.gz 2023-06-23T18:37:54.044 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114600.log 2023-06-23T18:37:54.044 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112011.log 2023-06-23T18:37:54.044 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111873.log 2023-06-23T18:37:54.044 INFO:teuthology.orchestra.run.smithi079.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114600.log.gz 2023-06-23T18:37:54.044 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112011.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-06-23T18:37:54.045 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111873.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112011.log.gz 2023-06-23T18:37:54.045 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107032.log 2023-06-23T18:37:54.045 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111873.log.gz 2023-06-23T18:37:54.045 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116218.log 2023-06-23T18:37:54.045 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.107032.log: /var/log/ceph/ceph-client.admin.116218.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107032.log.gz 2023-06-23T18:37:54.046 INFO:teuthology.orchestra.run.smithi079.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116218.log.gz 2023-06-23T18:37:54.046 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119487.log 2023-06-23T18:37:54.046 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108665.log 2023-06-23T18:37:54.046 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-06-23T18:37:54.046 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115815.log 2023-06-23T18:37:54.047 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119487.log.gz 2023-06-23T18:37:54.047 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114508.log 2023-06-23T18:37:54.047 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108084.log 2023-06-23T18:37:54.047 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115815.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108665.log.gz 2023-06-23T18:37:54.047 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117639.log 2023-06-23T18:37:54.047 INFO:teuthology.orchestra.run.smithi079.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115815.log.gz 2023-06-23T18:37:54.048 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108084.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114508.log.gz 2023-06-23T18:37:54.048 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108084.log.gz 2023-06-23T18:37:54.048 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117639.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.117639.log.gz 2023-06-23T18:37:54.048 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119717.log 2023-06-23T18:37:54.048 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113213.log 2023-06-23T18:37:54.049 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106978.log 2023-06-23T18:37:54.049 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113213.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119717.log.gz 2023-06-23T18:37:54.049 INFO:teuthology.orchestra.run.smithi079.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113213.log.gz 2023-06-23T18:37:54.049 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.106978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115769.log 2023-06-23T18:37:54.049 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose 88.8%-- /var/log/ceph/ceph-client.admin.106736.log -- replaced with /var/log/ceph/ceph-client.admin.106978.log.gz 2023-06-23T18:37:54.050 INFO:teuthology.orchestra.run.smithi079.stderr: 2023-06-23T18:37:54.050 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113389.log 2023-06-23T18:37:54.050 INFO:teuthology.orchestra.run.smithi079.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115769.log.gz 2023-06-23T18:37:54.050 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.106736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114830.log 2023-06-23T18:37:54.050 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106277.log 2023-06-23T18:37:54.051 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113389.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106736.log.gz 2023-06-23T18:37:54.051 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116402.log 2023-06-23T18:37:54.051 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113389.log.gz 2023-06-23T18:37:54.051 INFO:teuthology.orchestra.run.smithi079.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114830.log.gz 2023-06-23T18:37:54.051 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.106277.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.106277.log.gz 2023-06-23T18:37:54.052 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100355.log 2023-06-23T18:37:54.052 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106214.log 2023-06-23T18:37:54.052 INFO:teuthology.orchestra.run.smithi079.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116402.log.gz 2023-06-23T18:37:54.052 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.100355.log: /var/log/ceph/ceph-client.admin.106214.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100355.log.gz 2023-06-23T18:37:54.052 INFO:teuthology.orchestra.run.smithi079.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106214.log.gz 2023-06-23T18:37:54.053 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108078.log 2023-06-23T18:37:54.053 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117908.log 2023-06-23T18:37:54.053 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107078.log 2023-06-23T18:37:54.053 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117908.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.108078.log.gz 2023-06-23T18:37:54.053 INFO:teuthology.orchestra.run.smithi079.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117908.log.gz 2023-06-23T18:37:54.054 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107078.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107078.log.gz 2023-06-23T18:37:54.054 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114278.log 2023-06-23T18:37:54.054 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116003.log 2023-06-23T18:37:54.054 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106690.log 2023-06-23T18:37:54.054 INFO:teuthology.orchestra.run.smithi079.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.114278.log.gz 2023-06-23T18:37:54.055 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110264.log 2023-06-23T18:37:54.055 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109355.log 2023-06-23T18:37:54.055 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.106690.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116003.log.gz 2023-06-23T18:37:54.055 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.110264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112349.log 2023-06-23T18:37:54.055 INFO:teuthology.orchestra.run.smithi079.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106690.log.gz 2023-06-23T18:37:54.056 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109355.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110264.log.gz 2023-06-23T18:37:54.056 INFO:teuthology.orchestra.run.smithi079.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.109355.log.gz 2023-06-23T18:37:54.056 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112349.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112349.log.gz 2023-06-23T18:37:54.056 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119832.log 2023-06-23T18:37:54.056 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106643.log 2023-06-23T18:37:54.057 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118000.log 2023-06-23T18:37:54.063 INFO:teuthology.orchestra.run.smithi155.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-06-23T18:37:54.069 INFO:teuthology.orchestra.run.smithi155.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-06-23T18:37:54.069 INFO:teuthology.orchestra.run.smithi155.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-06-23T18:37:54.076 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.106643.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119832.log.gz 2023-06-23T18:37:54.077 INFO:teuthology.orchestra.run.smithi079.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106643.log.gz 2023-06-23T18:37:54.077 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118000.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118000.log.gz 2023-06-23T18:37:54.077 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113619.log 2023-06-23T18:37:54.077 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114140.log 2023-06-23T18:37:54.077 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113868.log 2023-06-23T18:37:54.078 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114140.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113619.log.gz 2023-06-23T18:37:54.078 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114140.log.gz 2023-06-23T18:37:54.078 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113868.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.113868.log.gz 2023-06-23T18:37:54.078 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109559.log 2023-06-23T18:37:54.078 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-06-23T18:37:54.079 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113527.log 2023-06-23T18:37:54.079 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.1.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109559.log.gz 2023-06-23T18:37:54.079 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113527.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113527.log.gz 2023-06-23T18:37:54.079 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112937.log 2023-06-23T18:37:54.079 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117685.log 2023-06-23T18:37:54.080 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112937.log: /var/log/ceph/ceph-client.admin.117685.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112937.log.gz 2023-06-23T18:37:54.080 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112845.log 2023-06-23T18:37:54.080 INFO:teuthology.orchestra.run.smithi079.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117685.log.gz 2023-06-23T18:37:54.080 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113914.log 2023-06-23T18:37:54.080 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112845.log: /var/log/ceph/ceph-client.admin.113914.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112845.log.gz 2023-06-23T18:37:54.081 INFO:teuthology.orchestra.run.smithi079.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.113914.log.gz 2023-06-23T18:37:54.081 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116310.log 2023-06-23T18:37:54.081 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106325.log 2023-06-23T18:37:54.081 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111735.log 2023-06-23T18:37:54.081 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.106325.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116310.log.gz 2023-06-23T18:37:54.082 INFO:teuthology.orchestra.run.smithi079.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106325.log.gz 2023-06-23T18:37:54.082 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111735.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111735.log.gz 2023-06-23T18:37:54.082 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108083.log 2023-06-23T18:37:54.082 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107264.log 2023-06-23T18:37:54.082 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108083.log: /var/log/ceph/ceph-client.admin.107264.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108083.log.gz 2023-06-23T18:37:54.083 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108895.log 2023-06-23T18:37:54.083 INFO:teuthology.orchestra.run.smithi079.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107264.log.gz 2023-06-23T18:37:54.083 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116610.log 2023-06-23T18:37:54.083 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108895.log: /var/log/ceph/ceph-client.admin.116610.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108895.log.gz 2023-06-23T18:37:54.083 INFO:teuthology.orchestra.run.smithi079.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116610.log.gz 2023-06-23T18:37:54.084 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115217.log 2023-06-23T18:37:54.084 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119395.log 2023-06-23T18:37:54.084 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115217.log: /var/log/ceph/ceph-client.admin.119395.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115217.log.gz 2023-06-23T18:37:54.084 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119395.log.gz 2023-06-23T18:37:54.084 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119073.log 2023-06-23T18:37:54.085 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113305.log 2023-06-23T18:37:54.085 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115447.log 2023-06-23T18:37:54.085 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113305.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119073.log.gz 2023-06-23T18:37:54.085 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115447.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113305.log.gz 2023-06-23T18:37:54.085 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111689.log 2023-06-23T18:37:54.086 INFO:teuthology.orchestra.run.smithi079.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115447.log.gz 2023-06-23T18:37:54.086 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111597.log 2023-06-23T18:37:54.086 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111689.log: /var/log/ceph/ceph-client.admin.111597.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111689.log.gz 2023-06-23T18:37:54.086 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111597.log.gz 2023-06-23T18:37:54.086 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113481.log 2023-06-23T18:37:54.087 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116540.log 2023-06-23T18:37:54.087 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117106.log 2023-06-23T18:37:54.087 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116540.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113481.log.gz 2023-06-23T18:37:54.087 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117106.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117106.log.gz 2023-06-23T18:37:54.087 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108987.log 2023-06-23T18:37:54.088 INFO:teuthology.orchestra.run.smithi079.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.116540.log.gz 2023-06-23T18:37:54.088 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115677.log 2023-06-23T18:37:54.088 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108987.log: /var/log/ceph/ceph-client.admin.115677.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108987.log.gz 2023-06-23T18:37:54.088 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115677.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115723.log 2023-06-23T18:37:54.088 INFO:teuthology.orchestra.run.smithi079.stderr: 2023-06-23T18:37:54.089 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100665.log 2023-06-23T18:37:54.089 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115723.log: /var/log/ceph/ceph-client.admin.100665.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115723.log.gz 2023-06-23T18:37:54.089 INFO:teuthology.orchestra.run.smithi079.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100665.log.gz 2023-06-23T18:37:54.089 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112211.log 2023-06-23T18:37:54.089 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108711.log 2023-06-23T18:37:54.090 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108081.log 2023-06-23T18:37:54.090 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108711.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112211.log.gz 2023-06-23T18:37:54.090 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108081.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108711.log.gz 2023-06-23T18:37:54.090 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108081.log.gz 2023-06-23T18:37:54.090 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119533.log 2023-06-23T18:37:54.090 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107494.log 2023-06-23T18:37:54.091 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119533.log: /var/log/ceph/ceph-client.admin.107494.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119533.log.gz 2023-06-23T18:37:54.091 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107494.log.gz 2023-06-23T18:37:54.091 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116356.log 2023-06-23T18:37:54.091 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118046.log 2023-06-23T18:37:54.091 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116356.log: /var/log/ceph/ceph-client.admin.118046.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.116356.log.gz 2023-06-23T18:37:54.092 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118046.log.gz 2023-06-23T18:37:54.092 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111781.log 2023-06-23T18:37:54.092 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113029.log 2023-06-23T18:37:54.092 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111781.log: /var/log/ceph/ceph-client.admin.113029.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111781.log.gz 2023-06-23T18:37:54.092 INFO:teuthology.orchestra.run.smithi079.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.113029.log.gz 2023-06-23T18:37:54.093 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112395.log 2023-06-23T18:37:54.093 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118322.log 2023-06-23T18:37:54.093 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108452.log 2023-06-23T18:37:54.093 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118322.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112395.log.gz 2023-06-23T18:37:54.093 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108452.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118322.log.gz 2023-06-23T18:37:54.094 INFO:teuthology.orchestra.run.smithi079.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108452.log.gz 2023-06-23T18:37:54.094 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118184.log 2023-06-23T18:37:54.094 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119441.log 2023-06-23T18:37:54.094 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118184.log: /var/log/ceph/ceph-client.admin.119441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109079.log 2023-06-23T18:37:54.094 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118184.log.gz 2023-06-23T18:37:54.095 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106828.log 2023-06-23T18:37:54.095 INFO:teuthology.orchestra.run.smithi079.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119441.log.gz 2023-06-23T18:37:54.095 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118843.log 2023-06-23T18:37:54.095 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.106828.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109079.log.gz 2023-06-23T18:37:54.095 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118843.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106828.log.gz 2023-06-23T18:37:54.096 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118843.log.gz 2023-06-23T18:37:54.096 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112891.log 2023-06-23T18:37:54.096 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117823.log 2023-06-23T18:37:54.096 INFO:teuthology.orchestra.run.smithi079.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-06-23T18:37:54.096 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115493.log 2023-06-23T18:37:54.097 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117823.log: gzip -5 --verbose -- 88.7%/var/log/ceph/ceph-client.admin.114922.log 2023-06-23T18:37:54.097 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112891.log.gz 2023-06-23T18:37:54.097 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117823.log.gz 2023-06-23T18:37:54.097 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115493.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115493.log.gz 2023-06-23T18:37:54.097 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114922.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114922.log.gz 2023-06-23T18:37:54.098 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108456.log 2023-06-23T18:37:54.098 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111965.log 2023-06-23T18:37:54.098 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115585.log 2023-06-23T18:37:54.098 INFO:teuthology.orchestra.run.smithi079.stderr: 87.7%/var/log/ceph/ceph-client.admin.111965.log: -- replaced with /var/log/ceph/ceph-client.admin.108456.log.gz 2023-06-23T18:37:54.098 INFO:teuthology.orchestra.run.smithi079.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111965.log.gz 2023-06-23T18:37:54.099 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115585.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.115585.log.gz 2023-06-23T18:37:54.099 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107172.log 2023-06-23T18:37:54.099 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118414.log 2023-06-23T18:37:54.099 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111413.log 2023-06-23T18:37:54.099 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118414.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107172.log.gz 2023-06-23T18:37:54.100 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118414.log.gz 2023-06-23T18:37:54.100 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111413.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111413.log.gz 2023-06-23T18:37:54.100 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119671.log 2023-06-23T18:37:54.100 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.94072.log 2023-06-23T18:37:54.100 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114646.log 2023-06-23T18:37:54.101 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.tmp-client.admin.94072.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119671.log.gz 2023-06-23T18:37:54.101 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.94072.log.gz 2023-06-23T18:37:54.101 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114646.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114646.log.gz 2023-06-23T18:37:54.101 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113573.log 2023-06-23T18:37:54.101 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119027.log 2023-06-23T18:37:54.102 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118368.log 2023-06-23T18:37:54.102 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119027.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113573.log.gz 2023-06-23T18:37:54.102 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118368.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119027.log.gz 2023-06-23T18:37:54.102 INFO:teuthology.orchestra.run.smithi079.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118368.log.gz 2023-06-23T18:37:54.102 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116448.log 2023-06-23T18:37:54.103 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107721.log 2023-06-23T18:37:54.103 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114186.log 2023-06-23T18:37:54.103 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107721.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116448.log.gz 2023-06-23T18:37:54.103 INFO:teuthology.orchestra.run.smithi079.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107721.log.gz 2023-06-23T18:37:54.103 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114186.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114186.log.gz 2023-06-23T18:37:54.103 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109401.log 2023-06-23T18:37:54.104 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119257.log 2023-06-23T18:37:54.104 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118276.log 2023-06-23T18:37:54.104 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119257.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109401.log.gz 2023-06-23T18:37:54.104 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119257.log.gz 2023-06-23T18:37:54.104 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118276.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.118276.log.gz 2023-06-23T18:37:54.105 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115539.log 2023-06-23T18:37:54.105 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112257.log 2023-06-23T18:37:54.105 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111505.log 2023-06-23T18:37:54.105 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112257.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115539.log.gz 2023-06-23T18:37:54.105 INFO:teuthology.orchestra.run.smithi079.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112257.log.gz 2023-06-23T18:37:54.106 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111505.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111505.log.gz 2023-06-23T18:37:54.106 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115861.log 2023-06-23T18:37:54.106 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114370.log 2023-06-23T18:37:54.106 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108455.log 2023-06-23T18:37:54.106 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117505.log 2023-06-23T18:37:54.107 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115861.log.gz 2023-06-23T18:37:54.107 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108090.log 2023-06-23T18:37:54.107 INFO:teuthology.orchestra.run.smithi079.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.114370.log.gz 2023-06-23T18:37:54.107 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117954.log 2023-06-23T18:37:54.107 INFO:teuthology.orchestra.run.smithi079.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.108455.log.gz 2023-06-23T18:37:54.108 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108090.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117505.log.gz 2023-06-23T18:37:54.108 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108090.log.gz 2023-06-23T18:37:54.108 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117954.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117954.log.gz 2023-06-23T18:37:54.108 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-06-23T18:37:54.108 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112165.log 2023-06-23T18:37:54.109 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115631.log 2023-06-23T18:37:54.109 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112303.log 2023-06-23T18:37:54.109 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115631.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112165.log.gz 2023-06-23T18:37:54.109 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112303.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115631.log.gz 2023-06-23T18:37:54.109 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112303.log.gz 2023-06-23T18:37:54.110 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107722.log 2023-06-23T18:37:54.110 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114738.log 2023-06-23T18:37:54.110 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112057.log 2023-06-23T18:37:54.110 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114738.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107722.log.gz 2023-06-23T18:37:54.110 INFO:teuthology.orchestra.run.smithi079.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.114738.log.gz 2023-06-23T18:37:54.111 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112057.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112057.log.gz 2023-06-23T18:37:54.111 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117731.log 2023-06-23T18:37:54.111 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108457.log 2023-06-23T18:37:54.111 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107720.log 2023-06-23T18:37:54.111 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108457.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117731.log.gz 2023-06-23T18:37:54.111 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.107720.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.108457.log.gz 2023-06-23T18:37:54.112 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107720.log.gz 2023-06-23T18:37:54.112 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109033.log 2023-06-23T18:37:54.112 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111321.log 2023-06-23T18:37:54.112 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109033.log: /var/log/ceph/ceph-client.admin.111321.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109033.log.gz 2023-06-23T18:37:54.113 INFO:teuthology.orchestra.run.smithi079.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.111321.log.gz 2023-06-23T18:37:54.113 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118478.log 2023-06-23T18:37:54.113 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116103.log 2023-06-23T18:37:54.113 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118478.log: /var/log/ceph/ceph-client.admin.116103.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.118478.log.gz 2023-06-23T18:37:54.113 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose 89.2%-- /var/log/ceph/ceph-client.admin.100800.log -- replaced with /var/log/ceph/ceph-client.admin.116103.log.gz 2023-06-23T18:37:54.113 INFO:teuthology.orchestra.run.smithi079.stderr: 2023-06-23T18:37:54.114 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113822.log 2023-06-23T18:37:54.114 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.100800.log: /var/log/ceph/ceph-client.admin.113822.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100800.log.gz 2023-06-23T18:37:54.114 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113822.log.gz 2023-06-23T18:37:54.114 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-06-23T18:37:54.114 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113730.log 2023-06-23T18:37:54.115 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.116927.log 2023-06-23T18:37:54.115 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113730.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113730.log.gz/var/log/ceph/ceph-client.0.116927.log: 2023-06-23T18:37:54.115 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108449.log 2023-06-23T18:37:54.115 INFO:teuthology.orchestra.run.smithi079.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.116927.log.gz 2023-06-23T18:37:54.115 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107719.log 2023-06-23T18:37:54.116 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108449.log: /var/log/ceph/ceph-client.admin.107719.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108449.log.gz 2023-06-23T18:37:54.116 INFO:teuthology.orchestra.run.smithi079.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107719.log.gz 2023-06-23T18:37:54.116 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109125.log 2023-06-23T18:37:54.116 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109125.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109125.log.gz 2023-06-23T18:37:54.116 INFO:teuthology.orchestra.run.smithi079.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-06-23T18:37:54.117 INFO:teuthology.orchestra.run.smithi079.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-06-23T18:37:54.117 INFO:teuthology.orchestra.run.smithi079.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-06-23T18:37:54.118 INFO:teuthology.orchestra.run.smithi079.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-06-23T18:37:54.119 INFO:teuthology.orchestra.run.smithi155.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-06-23T18:37:54.119 INFO:teuthology.orchestra.run.smithi155.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-06-23T18:37:54.119 INFO:teuthology.orchestra.run.smithi155.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-06-23T18:37:54.119 INFO:teuthology.orchestra.run.smithi155.stderr: 2023-06-23T18:37:54.120 INFO:teuthology.orchestra.run.smithi155.stderr:real 0m0.153s 2023-06-23T18:37:54.120 INFO:teuthology.orchestra.run.smithi155.stderr:user 0m0.650s 2023-06-23T18:37:54.120 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m0.059s 2023-06-23T18:37:54.221 INFO:teuthology.orchestra.run.smithi079.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-06-23T18:37:54.222 INFO:teuthology.orchestra.run.smithi079.stderr: 2023-06-23T18:37:54.223 INFO:teuthology.orchestra.run.smithi079.stderr:real 0m0.298s 2023-06-23T18:37:54.223 INFO:teuthology.orchestra.run.smithi079.stderr:user 0m0.774s 2023-06-23T18:37:54.224 INFO:teuthology.orchestra.run.smithi079.stderr:sys 0m0.158s 2023-06-23T18:37:54.224 INFO:tasks.ceph:Archiving logs... 2023-06-23T18:37:54.225 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886/remote/smithi079/log 2023-06-23T18:37:54.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-23T18:37:54.581 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886/remote/smithi155/log 2023-06-23T18:37:54.582 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-23T18:37:54.684 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-06-23T18:37:54.711 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-06-23T18:37:54.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-23T18:37:54.720 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-23T18:37:54.800 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-06-23T18:37:54.810 INFO:teuthology.task.clock:Checking final clock skew... 2023-06-23T18:37:54.811 DEBUG:teuthology.orchestra.run.smithi079:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T18:37:54.813 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T18:37:54.827 INFO:teuthology.orchestra.run.smithi079.stderr:bash: ntpq: command not found 2023-06-23T18:37:54.829 INFO:teuthology.orchestra.run.smithi155.stderr:bash: ntpq: command not found 2023-06-23T18:37:54.832 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T18:37:54.832 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2023-06-23T18:37:54.832 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 65 -952us[-1091us] +/- 74ms 2023-06-23T18:37:54.833 INFO:teuthology.orchestra.run.smithi079.stdout:^* hv02.front.sepia.ceph.com 3 6 377 64 -5357us[-5496us] +/- 73ms 2023-06-23T18:37:54.833 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +5989us[+5989us] +/- 70ms 2023-06-23T18:37:54.833 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:37:54.834 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T18:37:54.834 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2023-06-23T18:37:54.835 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 0 -851us[ -851us] +/- 75ms 2023-06-23T18:37:54.835 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv02.front.sepia.ceph.com 3 6 377 0 -5318us[-5391us] +/- 74ms 2023-06-23T18:37:54.835 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +6041us[+5968us] +/- 70ms 2023-06-23T18:37:54.835 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:37:54.836 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-06-23T18:37:54.845 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-06-23T18:37:54.845 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-06-23T18:37:54.854 DEBUG:teuthology.orchestra.run.smithi079:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-23T18:37:54.972 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-23T18:37:55.063 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T18:37:55.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:37:55.096 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T18:37:55.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:37:55.129 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-06-23T18:37:55.139 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-06-23T18:37:55.150 INFO:teuthology.task.internal:Duration was 840.445563 seconds 2023-06-23T18:37:55.150 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-06-23T18:37:55.159 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-06-23T18:37:55.160 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-23T18:37:55.163 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-23T18:37:55.207 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T18:37:55.215 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T18:37:55.540 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-06-23T18:37:55.540 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi079.front.sepia.ceph.com 2023-06-23T18:37:55.541 DEBUG:teuthology.orchestra.run.smithi079:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-23T18:37:55.564 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2023-06-23T18:37:55.565 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-23T18:37:55.587 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-06-23T18:37:55.587 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T18:37:55.607 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T18:37:55.668 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-06-23T18:37:55.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-23T18:37:55.690 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-23T18:37:55.786 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-06-23T18:37:55.799 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-06-23T18:37:55.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-23T18:37:55.826 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-23T18:37:55.853 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-06-23T18:37:55.867 DEBUG:teuthology.orchestra.run.smithi079:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-23T18:37:55.869 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-23T18:37:55.892 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = core 2023-06-23T18:37:55.919 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2023-06-23T18:37:55.933 DEBUG:teuthology.orchestra.run.smithi079:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-23T18:37:55.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:37:55.962 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-23T18:37:55.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:37:55.991 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-06-23T18:37:56.004 INFO:teuthology.task.internal:Transferring archived files... 2023-06-23T18:37:56.006 DEBUG:teuthology.misc:Transferring archived files from smithi079:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886/remote/smithi079 2023-06-23T18:37:56.007 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-23T18:37:56.043 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886/remote/smithi155 2023-06-23T18:37:56.044 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-23T18:37:56.079 INFO:teuthology.task.internal:Removing archive directory... 2023-06-23T18:37:56.079 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-23T18:37:56.081 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-23T18:37:56.131 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-06-23T18:37:56.144 INFO:teuthology.task.internal:Not uploading archives. 2023-06-23T18:37:56.145 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-06-23T18:37:56.154 INFO:teuthology.task.internal:Tidying up after the test... 2023-06-23T18:37:56.155 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-23T18:37:56.157 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-23T18:37:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 23 18:37 /home/ubuntu/cephtest 2023-06-23T18:37:56.187 INFO:teuthology.orchestra.run.smithi155.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 23 18:37 /home/ubuntu/cephtest 2023-06-23T18:37:56.189 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-06-23T18:37:56.202 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-06-23T18:37:56.239 INFO:teuthology.nuke:Checking targets against current locks 2023-06-23T18:37:56.269 DEBUG:teuthology.nuke:shortname: smithi079 2023-06-23T18:37:56.270 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T18:37:56.294 DEBUG:teuthology.nuke:shortname: smithi155 2023-06-23T18:37:56.295 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T18:37:56.298 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 18:18:21.501393', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-06-23T18:37:56.328 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313886', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 18:18:21.503336', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-23T18:37:56.367 INFO:teuthology.orchestra.console:Power off smithi079 2023-06-23T18:37:56.369 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-23T18:37:56.387 INFO:teuthology.orchestra.console:Power off smithi155 2023-06-23T18:37:56.388 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-23T18:37:56.400 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T18:37:56.412 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T18:38:00.425 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T18:38:00.438 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T18:38:04.540 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T18:38:04.553 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T18:38:08.656 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T18:38:08.665 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T18:38:08.869 INFO:teuthology.orchestra.console:Power off for smithi079 completed 2023-06-23T18:38:08.878 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2023-06-23T18:38:08.984 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} duration: 840.4455633163452 failure_reason: 'Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=ad270baecd2e4f3485a40bd8fa4a0a47 status: fail success: false 2023-06-23T18:38:08.985 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-23T18:38:09.062 INFO:teuthology.run:FAIL